Jul 16 12:39:06.649361 ip-10-0-140-165 systemd[1]: Starting Kubernetes Kubelet... Jul 16 12:39:07.059836 ip-10-0-140-165 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 12:39:07.059836 ip-10-0-140-165 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 12:39:07.059836 ip-10-0-140-165 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 12:39:07.059836 ip-10-0-140-165 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 12:39:07.059836 ip-10-0-140-165 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 12:39:07.059836 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.044022 2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 12:39:07.059836 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048324 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:39:07.059836 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048339 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:39:07.059836 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048343 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:39:07.059836 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048347 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:39:07.059836 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048350 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:39:07.059836 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048353 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:39:07.059836 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048355 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:39:07.059836 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048358 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:39:07.059836 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048361 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048366 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048372 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048375 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048378 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048381 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048384 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048389 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048393 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048396 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048399 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048401 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048404 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048407 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048409 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048413 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048415 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048418 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048421 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:39:07.061554 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048424 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048426 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048429 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048432 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048434 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048438 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048441 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048443 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048446 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048448 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048451 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048456 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048458 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048462 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048464 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048468 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048471 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048474 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048477 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:39:07.062201 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048480 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048482 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048487 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048489 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048492 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048494 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048497 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048499 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048502 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048506 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048509 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048512 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048515 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048517 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048521 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048524 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048527 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048529 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048532 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048535 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:39:07.063501 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048538 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048541 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048544 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048546 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048549 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048552 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048555 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048558 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048560 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048563 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048565 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048568 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048570 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048573 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048575 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048578 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048580 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048583 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048585 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048588 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:39:07.064359 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048591 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048594 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048596 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048599 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048602 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048604 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048607 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048609 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048612 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048615 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048617 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048620 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.048623 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049354 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049362 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049365 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049368 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049373 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049377 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:39:07.065018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049380 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049383 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049386 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049389 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049392 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049394 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049397 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049400 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049403 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049407 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049410 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049412 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049415 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049418 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049420 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049423 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049426 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049428 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049432 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049434 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:39:07.065914 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049438 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049440 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049443 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049446 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049449 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049452 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049454 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049457 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049459 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049463 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049465 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049468 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049471 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049474 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049476 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049479 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049481 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049484 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049486 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049489 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:39:07.066827 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049492 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049494 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049497 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049499 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049502 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049505 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049507 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049509 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049512 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049516 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049518 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049520 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049524 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049526 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049529 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049533 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049537 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049540 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049543 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049546 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:39:07.067935 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049548 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049551 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049554 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049557 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049560 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049563 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049566 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049568 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049571 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049573 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049575 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049578 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049580 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049583 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049585 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049588 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049590 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049592 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049595 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049597 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:39:07.068588 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049600 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049602 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049605 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049608 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049611 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049614 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049616 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049619 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049623 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049627 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049630 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049632 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.049635 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050084 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050091 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050094 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050097 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050104 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050107 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:39:07.069245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050110 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050114 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050117 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050120 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050122 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050125 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050128 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050130 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050133 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050136 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050140 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050144 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050151 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050155 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050158 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050161 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050165 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050168 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:39:07.069903 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050171 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050174 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050176 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050179 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050181 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050184 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050187 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050189 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050192 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050194 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050197 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050200 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050202 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050205 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050208 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050210 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050213 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050216 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050218 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050221 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:39:07.070561 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050223 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050226 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050229 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050231 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050234 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050237 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050241 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050244 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050246 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050249 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050251 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050254 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050256 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050259 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050261 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050264 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050266 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050269 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050272 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050274 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:39:07.071426 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050277 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050279 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050281 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050284 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050286 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050289 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050293 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050297 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050299 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050302 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050304 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050307 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050309 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050312 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050314 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050317 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050319 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050322 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050326 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050328 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:39:07.072307 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050331 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050333 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050336 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050338 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050342 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050346 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050349 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050352 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050362 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050365 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050368 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050370 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050373 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050376 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.050378 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050446 2567 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050455 2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050462 2567 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050467 2567 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050473 2567 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050477 2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 12:39:07.073612 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050481 2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050485 2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050488 2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050491 2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050494 2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050498 2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050501 2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050504 2567 flags.go:64] FLAG: --cgroup-root="" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050507 2567 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050512 2567 flags.go:64] FLAG: --client-ca-file="" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050514 2567 flags.go:64] FLAG: --cloud-provider="external" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050517 2567 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050521 2567 flags.go:64] FLAG: --cluster-domain="" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050524 2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050528 2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050531 2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050534 2567 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050538 2567 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050541 2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050544 2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050547 2567 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050550 2567 flags.go:64] FLAG: --contention-profiling="false" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050553 2567 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050556 2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 12:39:07.074265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050560 2567 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050562 2567 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050566 2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050569 2567 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050572 2567 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050575 2567 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050580 2567 flags.go:64] FLAG: --enable-server="true" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050584 2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050607 2567 flags.go:64] FLAG: --event-burst="100" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050611 2567 flags.go:64] FLAG: --event-qps="50" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050614 2567 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050618 2567 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050621 2567 flags.go:64] FLAG: --eviction-hard="" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050625 2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050627 2567 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050630 2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050633 2567 flags.go:64] FLAG: --eviction-soft="" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050643 2567 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050647 2567 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050650 2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050652 2567 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050655 2567 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050658 2567 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050661 2567 flags.go:64] FLAG: --feature-gates="" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050665 2567 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 12:39:07.076009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050668 2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050671 2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050674 2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050677 2567 flags.go:64] FLAG: --healthz-port="10248" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050680 2567 flags.go:64] FLAG: --help="false" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050683 2567 flags.go:64] FLAG: --hostname-override="ip-10-0-140-165.ec2.internal" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050686 2567 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050690 2567 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050692 2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050696 2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050699 2567 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050702 2567 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050705 2567 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050709 2567 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050712 2567 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050715 2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050719 2567 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050722 2567 flags.go:64] FLAG: --kube-reserved="" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050725 2567 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050727 2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050731 2567 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050733 2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050736 2567 flags.go:64] FLAG: --lock-file="" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050739 2567 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 12:39:07.076876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050744 2567 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050747 2567 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050752 2567 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050755 2567 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050758 2567 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050761 2567 flags.go:64] FLAG: --logging-format="text" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050763 2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050767 2567 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050769 2567 flags.go:64] FLAG: --manifest-url="" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050772 2567 flags.go:64] FLAG: --manifest-url-header="" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050776 2567 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050779 2567 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050783 2567 flags.go:64] FLAG: --max-pods="110" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050786 2567 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050789 2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050792 2567 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050795 2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050798 2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050801 2567 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050804 2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050812 2567 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050816 2567 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050819 2567 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 12:39:07.077989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050823 2567 flags.go:64] FLAG: --pod-cidr="" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050826 2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050832 2567 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050834 2567 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050838 2567 flags.go:64] FLAG: --pods-per-core="0" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050841 2567 flags.go:64] FLAG: --port="10250" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050844 2567 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050846 2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05c1d0c616e67d4d2" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050850 2567 flags.go:64] FLAG: --qos-reserved="" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050854 2567 flags.go:64] FLAG: --read-only-port="10255" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050857 2567 flags.go:64] FLAG: --register-node="true" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050860 2567 flags.go:64] FLAG: --register-with-taints="" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050863 2567 flags.go:64] FLAG: --registry-burst="10" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050866 2567 flags.go:64] FLAG: --registry-qps="5" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050869 2567 flags.go:64] FLAG: --reserved-cpus="" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050872 2567 flags.go:64] FLAG: --reserved-memory="" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050876 2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050879 2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050882 2567 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050885 2567 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050888 2567 flags.go:64] FLAG: --runonce="false" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050890 2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050893 2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050896 2567 flags.go:64] FLAG: --seccomp-default="false" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050899 2567 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 12:39:07.078862 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050902 2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050905 2567 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050908 2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050911 2567 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050914 2567 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050917 2567 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050921 2567 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050924 2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050927 2567 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050931 2567 flags.go:64] FLAG: --system-cgroups="" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050934 2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050939 2567 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050942 2567 flags.go:64] FLAG: --tls-cert-file="" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050945 2567 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050949 2567 flags.go:64] FLAG: --tls-min-version="" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050952 2567 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050956 2567 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050959 2567 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050962 2567 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050965 2567 flags.go:64] FLAG: --v="2" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050968 2567 flags.go:64] FLAG: --version="false" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050972 2567 flags.go:64] FLAG: --vmodule="" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050976 2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.050979 2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051141 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:39:07.079736 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051147 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051150 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051153 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051155 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051158 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051161 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051163 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051166 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051168 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051171 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051174 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051176 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051179 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051183 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051186 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051189 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051191 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051194 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051197 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051200 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:39:07.080623 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051203 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051206 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051209 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051212 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051215 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051218 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051220 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051223 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051225 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051228 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051230 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051233 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051235 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051238 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051240 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051243 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051245 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051248 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051251 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:39:07.081760 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051253 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051256 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051258 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051260 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051263 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051265 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051270 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051273 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051275 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051278 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051281 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051283 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051286 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051289 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051291 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051294 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051298 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051301 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051304 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051306 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:39:07.083590 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051309 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051311 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051315 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051319 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051322 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051325 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051327 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051330 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051334 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051336 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051339 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051341 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051344 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051346 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051349 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051352 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051354 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051357 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051361 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051363 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:39:07.084245 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051366 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051368 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051371 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051373 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051376 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051379 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051381 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051384 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051388 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051391 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051394 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051396 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051398 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051401 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051403 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051407 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051411 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051415 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:39:07.084934 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.051418 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:39:07.085613 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.051948 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 12:39:07.085613 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.058975 2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 12:39:07.085613 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.058994 2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 12:39:07.085613 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.059011 2567 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 12:39:07.085613 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.059020 2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 12:39:07.085613 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059120 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:39:07.085613 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059126 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:39:07.085613 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059130 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:39:07.085613 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059133 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:39:07.085613 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059136 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:39:07.085613 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059139 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:39:07.085613 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059142 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:39:07.085613 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059144 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:39:07.085613 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059147 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059150 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059153 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059155 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059158 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059160 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059163 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059165 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059168 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059170 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059179 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059182 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059184 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059187 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059189 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059192 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059194 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059197 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059199 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059202 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:39:07.139969 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059204 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:39:07.121737 ip-10-0-140-165 systemd[1]: Started Kubernetes Kubelet. Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059215 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059218 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059220 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059222 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059225 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059227 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059230 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059233 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059235 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059238 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059240 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059243 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059245 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059248 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059250 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059253 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059255 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059258 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059260 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:39:07.140793 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059263 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059265 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059268 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059270 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059273 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059277 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059281 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059284 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059287 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059290 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059293 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059296 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059298 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059306 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059318 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059322 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059326 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059329 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059333 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:39:07.141855 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059336 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059339 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059342 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059344 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059347 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059349 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059351 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059354 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059357 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059359 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059362 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059365 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059367 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059370 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059373 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059376 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059379 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059382 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059384 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059386 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:39:07.142508 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059389 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:39:07.143801 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059391 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:39:07.143801 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059394 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:39:07.143801 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059396 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:39:07.143801 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059398 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:39:07.143801 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059401 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:39:07.143801 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059404 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:39:07.143801 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059407 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:39:07.143801 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059415 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:39:07.143801 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059418 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:39:07.143801 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059420 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:39:07.143801 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059424 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:39:07.143801 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.059429 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 12:39:07.143801 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059583 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:39:07.143801 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059588 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:39:07.143801 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059591 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059594 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059596 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059599 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059601 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059604 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059607 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059609 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059612 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059614 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059617 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059619 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059622 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059625 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059627 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059630 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059632 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059635 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059638 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:39:07.144455 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059640 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059643 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059646 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059648 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059653 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059655 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059657 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059665 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059668 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059671 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059674 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059677 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059680 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059683 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059685 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059688 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059691 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059695 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059698 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059701 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:39:07.145755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059704 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059706 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059709 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059711 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059714 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059716 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059719 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059721 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059723 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059727 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059730 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059733 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059736 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059738 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059741 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059743 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059747 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059749 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059752 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:39:07.146403 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059755 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059763 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059766 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059770 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059772 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059775 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059777 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059780 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059782 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059785 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059787 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059790 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059792 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059795 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059797 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059800 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059802 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059805 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059807 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059809 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:39:07.147489 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059812 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059814 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059817 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059819 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059822 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059824 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059826 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059829 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059831 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059836 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059839 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059843 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059846 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059849 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059858 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059862 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059865 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059868 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:39:07.148351 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:07.059871 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:39:07.149896 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.059876 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 12:39:07.149896 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.060001 2567 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 12:39:07.149896 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.063432 2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 12:39:07.149896 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.064866 2567 server.go:1019] "Starting client certificate rotation" Jul 16 12:39:07.149896 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.064966 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 12:39:07.149896 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.066030 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 12:39:07.149896 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.070352 2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 12:39:07.149896 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.071508 2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 12:39:07.149896 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.086480 2567 log.go:25] "Validated CRI v1 runtime API" Jul 16 12:39:07.149896 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.088629 2567 log.go:25] "Validated CRI v1 image API" Jul 16 12:39:07.149896 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.090250 2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 12:39:07.149896 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.093549 2567 fs.go:135] Filesystem UUIDs: map[5a770369-2d72-4e3c-a751-71a03e5be9a3:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e9769161-6f99-4986-a1fd-c3bb2e8732b7:/dev/nvme0n1p4] Jul 16 12:39:07.150337 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.093564 2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 12:39:07.150337 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.098308 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.099186 2567 manager.go:216] Machine: {Timestamp:2026-07-16 12:39:07.098682569 +0000 UTC m=+0.332590411 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104985 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23a07f11904bef031d877d75196084 SystemUUID:ec23a07f-1190-4bef-031d-877d75196084 BootID:d0d40c8a-8faa-4f42-b3d0-56c5373d6846 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0e:ed:e2:5d:89 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0e:ed:e2:5d:89 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:90:b3:99:b7:39 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.099283 2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.099362 2567 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.102319 2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.102348 2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-165.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.103005 2567 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.103014 2567 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.103021 2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.103033 2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.104410 2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.107875 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.107985 2567 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.108012 2567 kubelet.go:486] "Attempting to sync node with API server" Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.108023 2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.108036 2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.108061 2567 kubelet.go:398] "Adding apiserver pod source" Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.108086 2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 12:39:07.150799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.108509 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.108525 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.110600 2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.115615 2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.115653 2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.115870 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.115895 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.115908 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.115953 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.115992 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.116002 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.116547 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.116559 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.116568 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.116574 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.116583 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.116592 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.117389 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.117399 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.119602 2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-165.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.119620 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.119620 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.120457 2567 server.go:1268] "Started kubelet" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.120519 2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.120596 2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.120668 2567 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.122983 2567 server.go:310] "Adding debug handlers to kubelet server" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.123264 2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.124935 2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.128901 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nwg6j" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.128972 2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.128992 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.129217 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.129766 2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.129780 2567 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.129794 2567 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.129899 2567 reconstruct.go:97] "Volume reconstruction finished" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.129910 2567 reconciler.go:29] "Reconciler: start to sync state" Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.130245 2567 factory.go:55] Registering systemd factory Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.130259 2567 factory.go:223] Registration of the systemd container factory successfully Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.130579 2567 factory.go:153] Registering CRI-O factory Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.130593 2567 factory.go:223] Registration of the crio container factory successfully Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.130642 2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.130669 2567 factory.go:103] Registering Raw factory Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.130687 2567 manager.go:1196] Started watching for new ooms in manager Jul 16 12:39:07.151865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.131210 2567 manager.go:349] Starting recovery of all containers Jul 16 12:39:07.153379 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.130553 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-165.ec2.internal.18c2c57df2650766 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-165.ec2.internal,UID:ip-10-0-140-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-165.ec2.internal,},FirstTimestamp:2026-07-16 12:39:07.120428902 +0000 UTC m=+0.354336743,LastTimestamp:2026-07-16 12:39:07.120428902 +0000 UTC m=+0.354336743,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-165.ec2.internal,}" Jul 16 12:39:07.153379 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.135951 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nwg6j" Jul 16 12:39:07.153379 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.141303 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 12:39:07.153379 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.143796 2567 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-165.ec2.internal\" not found" node="ip-10-0-140-165.ec2.internal" Jul 16 12:39:07.231022 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.230169 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found" Jul 16 12:39:07.354489 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.346982 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found" Jul 16 12:39:07.354489 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.347113 2567 manager.go:354] Recovery completed Jul 16 12:39:07.448756 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.447719 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found" Jul 16 12:39:07.521843 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.521823 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:39:07.524214 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.524191 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:39:07.524296 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.524228 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:39:07.524296 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.524240 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:39:07.524908 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.524889 2567 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 12:39:07.524908 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.524905 2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 12:39:07.525022 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.524918 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:39:07.525022 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.524930 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 16 12:39:07.525713 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.525695 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:39:07.525802 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.525726 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:39:07.525802 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.525743 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:39:07.526443 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.526429 2567 policy_none.go:49] "None policy: Start" Jul 16 12:39:07.526480 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.526448 2567 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 12:39:07.526480 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.526460 2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 12:39:07.527849 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.527828 2567 policy_none.go:47] "Start" Jul 16 12:39:07.548579 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.548429 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found" Jul 16 12:39:07.568763 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.568747 2567 manager.go:337] "Starting Device Plugin manager" Jul 16 12:39:07.568861 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.568779 2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 12:39:07.568861 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.568789 2567 server.go:87] "Starting device plugin registration server" Jul 16 12:39:07.569059 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.569046 2567 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 12:39:07.569136 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.569062 2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 12:39:07.569244 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.569218 2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 12:39:07.569330 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.569304 2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 12:39:07.569330 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.569317 2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 12:39:07.570308 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.570270 2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 12:39:07.570399 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.570333 2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-165.ec2.internal\" not found" Jul 16 12:39:07.631475 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.631442 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 12:39:07.632718 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.632703 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 12:39:07.632803 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.632739 2567 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 12:39:07.632803 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.632762 2567 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 12:39:07.632803 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.632794 2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 12:39:07.637103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.637083 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 12:39:07.669237 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.669192 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:39:07.670270 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.670255 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:39:07.670353 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.670291 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:39:07.670353 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.670306 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:39:07.670353 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.670337 2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-165.ec2.internal" Jul 16 12:39:07.687001 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.686977 2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-165.ec2.internal" Jul 16 12:39:07.687139 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.687005 2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-165.ec2.internal\": node \"ip-10-0-140-165.ec2.internal\" not found" Jul 16 12:39:07.720550 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.720526 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found" Jul 16 12:39:07.733250 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.733231 2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal"] Jul 16 12:39:07.733318 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.733308 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:39:07.734546 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.734530 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:39:07.734617 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.734565 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:39:07.734617 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.734580 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:39:07.735892 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.735879 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:39:07.736040 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.736025 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" Jul 16 12:39:07.736095 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.736071 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:39:07.736693 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.736678 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:39:07.736768 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.736702 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:39:07.736768 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.736717 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:39:07.737299 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.737274 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:39:07.737299 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.737303 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:39:07.737457 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.737313 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:39:07.738623 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.738607 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal" Jul 16 12:39:07.738677 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.738635 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:39:07.739410 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.739384 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:39:07.739410 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.739413 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:39:07.739530 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.739424 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:39:07.751740 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.751724 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-165.ec2.internal\" not found" node="ip-10-0-140-165.ec2.internal" Jul 16 12:39:07.755483 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.755467 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-165.ec2.internal\" not found" node="ip-10-0-140-165.ec2.internal" Jul 16 12:39:07.821310 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.821234 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found" Jul 16 12:39:07.852487 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.852457 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/33fcd2db8501242e604d3c0ecb5673f6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal\" (UID: \"33fcd2db8501242e604d3c0ecb5673f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" Jul 16 12:39:07.852487 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.852495 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33fcd2db8501242e604d3c0ecb5673f6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal\" (UID: \"33fcd2db8501242e604d3c0ecb5673f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" Jul 16 12:39:07.852677 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.852522 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/acfaee3e8f27ae7b52fdbaeafbb5fcbe-config\") pod \"kube-apiserver-proxy-ip-10-0-140-165.ec2.internal\" (UID: \"acfaee3e8f27ae7b52fdbaeafbb5fcbe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal" Jul 16 12:39:07.922092 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:07.922068 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found" Jul 16 12:39:07.952725 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.952696 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/33fcd2db8501242e604d3c0ecb5673f6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal\" (UID: \"33fcd2db8501242e604d3c0ecb5673f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" Jul 16 12:39:07.952812 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.952729 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33fcd2db8501242e604d3c0ecb5673f6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal\" (UID: \"33fcd2db8501242e604d3c0ecb5673f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" Jul 16 12:39:07.952812 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.952749 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/acfaee3e8f27ae7b52fdbaeafbb5fcbe-config\") pod \"kube-apiserver-proxy-ip-10-0-140-165.ec2.internal\" (UID: \"acfaee3e8f27ae7b52fdbaeafbb5fcbe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal" Jul 16 12:39:07.952812 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.952788 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/33fcd2db8501242e604d3c0ecb5673f6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal\" (UID: \"33fcd2db8501242e604d3c0ecb5673f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" Jul 16 12:39:07.952812 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.952802 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/acfaee3e8f27ae7b52fdbaeafbb5fcbe-config\") pod \"kube-apiserver-proxy-ip-10-0-140-165.ec2.internal\" (UID: \"acfaee3e8f27ae7b52fdbaeafbb5fcbe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal" Jul 16 12:39:07.952957 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:07.952801 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33fcd2db8501242e604d3c0ecb5673f6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal\" (UID: \"33fcd2db8501242e604d3c0ecb5673f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" Jul 16 12:39:08.027749 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:08.027695 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found" Jul 16 12:39:08.054339 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:08.054306 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" Jul 16 12:39:08.058716 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:08.058307 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal" Jul 16 12:39:08.064556 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:08.064535 2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 12:39:08.064952 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:08.064666 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 12:39:08.064952 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:08.064710 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 12:39:08.103926 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:08.103850 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 12:39:08.127883 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:08.127849 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found" Jul 16 12:39:08.138621 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:08.138586 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 12:34:07 +0000 UTC" deadline="2028-04-13 19:57:28.086966534 +0000 UTC" Jul 16 12:39:08.138621 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:08.138620 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15295h18m19.948349789s" Jul 16 12:39:08.229141 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:08.229107 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found" Jul 16 12:39:08.334220 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:08.334184 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found" Jul 16 12:39:08.434786 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:08.434758 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found" Jul 16 12:39:08.535960 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:08.535760 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found" Jul 16 12:39:08.595135 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:08.595096 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacfaee3e8f27ae7b52fdbaeafbb5fcbe.slice/crio-92adf780e1f5a87d47b2075ae52c24bae1d3b9a03bd6cae775c88a23052b34c2 WatchSource:0}: Error finding container 92adf780e1f5a87d47b2075ae52c24bae1d3b9a03bd6cae775c88a23052b34c2: Status 404 returned error can't find the container with id 92adf780e1f5a87d47b2075ae52c24bae1d3b9a03bd6cae775c88a23052b34c2 Jul 16 12:39:08.601281 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:08.601264 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 12:39:08.605591 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:08.605574 2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 12:39:08.630302 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:08.630270 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" Jul 16 12:39:08.636775 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:08.636736 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal" event={"ID":"acfaee3e8f27ae7b52fdbaeafbb5fcbe","Type":"ContainerStarted","Data":"92adf780e1f5a87d47b2075ae52c24bae1d3b9a03bd6cae775c88a23052b34c2"} Jul 16 12:39:08.641558 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:08.641543 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 12:39:08.642558 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:08.642545 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal" Jul 16 12:39:08.651501 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:08.651486 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 12:39:08.891370 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:08.891285 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 12:39:09.108453 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.108420 2567 apiserver.go:52] "Watching apiserver" Jul 16 12:39:09.117120 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.117079 2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 12:39:09.120376 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.120346 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-78h5z","kube-system/konnectivity-agent-6jbsz","kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6","openshift-cluster-node-tuning-operator/tuned-snw24","openshift-image-registry/node-ca-42h29","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal","openshift-multus/multus-additional-cni-plugins-v4lc5","openshift-dns/node-resolver-krt5w","openshift-multus/multus-lph7m","openshift-multus/network-metrics-daemon-d6kvg","openshift-network-diagnostics/network-check-target-sjhtn","openshift-network-operator/iptables-alerter-j7sfm","openshift-ovn-kubernetes/ovnkube-node-s729s"] Jul 16 12:39:09.121907 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.121882 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:09.122021 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.121975 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:09.123261 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.123238 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6jbsz" Jul 16 12:39:09.124625 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.124604 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.126015 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.125838 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.126327 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.126307 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-55nf8\"" Jul 16 12:39:09.126447 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.126311 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 12:39:09.126447 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.126310 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 12:39:09.127068 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.127033 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-42h29" Jul 16 12:39:09.127360 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.127341 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hmblm\"" Jul 16 12:39:09.127574 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.127556 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 12:39:09.127801 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.127782 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 12:39:09.127894 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.127872 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 12:39:09.128294 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.128263 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 12:39:09.128576 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.128545 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.128860 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.128708 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2w4h5\"" Jul 16 12:39:09.128860 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.128721 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 12:39:09.129697 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.129676 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-krt5w" Jul 16 12:39:09.129792 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.129701 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 12:39:09.130462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.130145 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 12:39:09.130462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.130164 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 12:39:09.130462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.130203 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 12:39:09.130462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.130171 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bthhj\"" Jul 16 12:39:09.131335 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.131163 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 12:39:09.131335 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.131261 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.131528 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.131508 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 12:39:09.131973 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.131774 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 12:39:09.131973 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.131837 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 12:39:09.131973 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.131847 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hdmlt\"" Jul 16 12:39:09.131973 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.131862 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 12:39:09.132930 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.132732 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:09.132930 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.132813 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:09.133115 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.132952 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 12:39:09.133115 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.133047 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-p8kkx\"" Jul 16 12:39:09.133115 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.133041 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 12:39:09.133888 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.133867 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 12:39:09.134824 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.134569 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-stqqp\"" Jul 16 12:39:09.134824 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.134632 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:09.134824 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.134702 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:09.136081 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.136021 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 12:39:09.136238 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.136208 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j7sfm" Jul 16 12:39:09.137626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.137608 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.138956 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.138844 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 12:39:09.138956 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.138894 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rzhqh\"" Jul 16 12:39:09.139155 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.138978 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 12:39:09.139155 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.138984 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 12:39:09.140754 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.140445 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 12:39:09.140754 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.140516 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 12:39:09.140754 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.140608 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 12:39:09.140754 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.140626 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 12:39:09.141009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.140762 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 12:39:09.141009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.140790 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gwvkk\"" Jul 16 12:39:09.142514 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.142451 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 12:39:09.144880 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.144744 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 12:39:09.158151 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158129 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-os-release\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.158275 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158253 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcl7f\" (UniqueName: \"kubernetes.io/projected/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-kube-api-access-dcl7f\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.158340 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158290 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-run-netns\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.158340 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158318 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-var-lib-kubelet\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.158451 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158342 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-etc-kubernetes\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.158451 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158367 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-sys-fs\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.158451 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158394 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-tuned\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.158451 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158417 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-multus-conf-dir\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.158451 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158443 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbmrj\" (UniqueName: \"kubernetes.io/projected/ae642324-8238-4122-8aa2-055642caafdd-kube-api-access-dbmrj\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.158665 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158468 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c9ad3582-7d79-470d-a2e0-cfcb46eef262-serviceca\") pod \"node-ca-42h29\" (UID: \"c9ad3582-7d79-470d-a2e0-cfcb46eef262\") " pod="openshift-image-registry/node-ca-42h29" Jul 16 12:39:09.158665 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158513 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-run-multus-certs\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.158665 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158574 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-socket-dir\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.158665 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158595 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-kubernetes\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.158665 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158622 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-sysctl-d\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.158665 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158644 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-run\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.158665 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158659 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a88360e6-cc58-4620-95c5-ce491044458c-tmp-dir\") pod \"node-resolver-krt5w\" (UID: \"a88360e6-cc58-4620-95c5-ce491044458c\") " pod="openshift-dns/node-resolver-krt5w" Jul 16 12:39:09.158942 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158676 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbqxr\" (UniqueName: \"kubernetes.io/projected/0b9829e7-da95-4c05-93b8-e494a91194e9-kube-api-access-xbqxr\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.158942 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158701 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-system-cni-dir\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.158942 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158727 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-cnibin\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.158942 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158759 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a91426dc-0372-43c1-9929-339429bd1d6a-konnectivity-ca\") pod \"konnectivity-agent-6jbsz\" (UID: \"a91426dc-0372-43c1-9929-339429bd1d6a\") " pod="kube-system/konnectivity-agent-6jbsz" Jul 16 12:39:09.158942 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158791 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c9ad3582-7d79-470d-a2e0-cfcb46eef262-host\") pod \"node-ca-42h29\" (UID: \"c9ad3582-7d79-470d-a2e0-cfcb46eef262\") " pod="openshift-image-registry/node-ca-42h29" Jul 16 12:39:09.158942 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158818 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5662m\" (UniqueName: \"kubernetes.io/projected/c9ad3582-7d79-470d-a2e0-cfcb46eef262-kube-api-access-5662m\") pod \"node-ca-42h29\" (UID: \"c9ad3582-7d79-470d-a2e0-cfcb46eef262\") " pod="openshift-image-registry/node-ca-42h29" Jul 16 12:39:09.158942 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158844 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.158942 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158865 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-multus-socket-dir-parent\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.158942 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158887 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-var-lib-cni-bin\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.158942 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158912 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a91426dc-0372-43c1-9929-339429bd1d6a-agent-certs\") pod \"konnectivity-agent-6jbsz\" (UID: \"a91426dc-0372-43c1-9929-339429bd1d6a\") " pod="kube-system/konnectivity-agent-6jbsz" Jul 16 12:39:09.158942 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158937 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-sysconfig\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.159472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158961 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-var-lib-kubelet\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.159472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.158999 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-os-release\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.159472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159034 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-multus-cni-dir\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.159472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159089 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ae642324-8238-4122-8aa2-055642caafdd-cni-binary-copy\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.159472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159116 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.159472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159141 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9l7v\" (UniqueName: \"kubernetes.io/projected/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-kube-api-access-n9l7v\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.159472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159167 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-systemd\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.159472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159197 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-device-dir\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.159472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159220 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-sys\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.159472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159261 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-host\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.159472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159300 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a88360e6-cc58-4620-95c5-ce491044458c-hosts-file\") pod \"node-resolver-krt5w\" (UID: \"a88360e6-cc58-4620-95c5-ce491044458c\") " pod="openshift-dns/node-resolver-krt5w" Jul 16 12:39:09.159472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159330 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-run-k8s-cni-cncf-io\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.159472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159363 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-var-lib-cni-multus\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.159472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159390 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-hostroot\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.159472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159424 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.159472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159472 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-system-cni-dir\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.160063 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159497 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ae642324-8238-4122-8aa2-055642caafdd-multus-daemon-config\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.160063 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159522 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b9829e7-da95-4c05-93b8-e494a91194e9-tmp\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.160063 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159547 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-etc-selinux\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.160063 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159605 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-lib-modules\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.160063 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159631 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-modprobe-d\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.160063 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159657 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.160063 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159681 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f0a26b92-3040-40c7-904f-7c8112a1f5e5-kubelet-config\") pod \"global-pull-secret-syncer-78h5z\" (UID: \"f0a26b92-3040-40c7-904f-7c8112a1f5e5\") " pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:09.160063 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159724 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-cnibin\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.160063 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159749 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f0a26b92-3040-40c7-904f-7c8112a1f5e5-dbus\") pod \"global-pull-secret-syncer-78h5z\" (UID: \"f0a26b92-3040-40c7-904f-7c8112a1f5e5\") " pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:09.160063 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159801 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwsm4\" (UniqueName: \"kubernetes.io/projected/a88360e6-cc58-4620-95c5-ce491044458c-kube-api-access-vwsm4\") pod \"node-resolver-krt5w\" (UID: \"a88360e6-cc58-4620-95c5-ce491044458c\") " pod="openshift-dns/node-resolver-krt5w" Jul 16 12:39:09.160063 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159827 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret\") pod \"global-pull-secret-syncer-78h5z\" (UID: \"f0a26b92-3040-40c7-904f-7c8112a1f5e5\") " pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:09.160063 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159845 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-registration-dir\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.160063 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159868 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-sysctl-conf\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.160063 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.159906 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.172899 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.172872 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tgj94" Jul 16 12:39:09.181011 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.180976 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tgj94" Jul 16 12:39:09.230954 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.230926 2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 12:39:09.260373 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260335 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcl7f\" (UniqueName: \"kubernetes.io/projected/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-kube-api-access-dcl7f\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.260580 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260381 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-run-netns\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.260580 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260416 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-systemd-units\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.260580 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260489 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-run-netns\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.260580 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260539 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-run-ovn\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.260580 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260571 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-cni-bin\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.260853 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260602 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-var-lib-kubelet\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.260853 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260624 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-etc-kubernetes\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.260853 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260640 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-sys-fs\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.260853 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260676 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-tuned\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.260853 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260706 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-var-lib-kubelet\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.260853 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260723 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-multus-conf-dir\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.260853 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260760 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbmrj\" (UniqueName: \"kubernetes.io/projected/ae642324-8238-4122-8aa2-055642caafdd-kube-api-access-dbmrj\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.260853 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260766 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-multus-conf-dir\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.260853 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260772 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-sys-fs\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.260853 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260705 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-etc-kubernetes\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.260853 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260790 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-env-overrides\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.260853 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260823 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c9ad3582-7d79-470d-a2e0-cfcb46eef262-serviceca\") pod \"node-ca-42h29\" (UID: \"c9ad3582-7d79-470d-a2e0-cfcb46eef262\") " pod="openshift-image-registry/node-ca-42h29" Jul 16 12:39:09.260853 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260850 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-run-multus-certs\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260879 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-run-netns\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260912 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b05d4e0-52bc-44b6-9409-f2597e4b9fd7-iptables-alerter-script\") pod \"iptables-alerter-j7sfm\" (UID: \"2b05d4e0-52bc-44b6-9409-f2597e4b9fd7\") " pod="openshift-network-operator/iptables-alerter-j7sfm" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260945 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d57m9\" (UniqueName: \"kubernetes.io/projected/2b05d4e0-52bc-44b6-9409-f2597e4b9fd7-kube-api-access-d57m9\") pod \"iptables-alerter-j7sfm\" (UID: \"2b05d4e0-52bc-44b6-9409-f2597e4b9fd7\") " pod="openshift-network-operator/iptables-alerter-j7sfm" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260973 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-socket-dir\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.260998 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-kubernetes\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261022 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-sysctl-d\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261027 2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261050 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-run\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261097 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a88360e6-cc58-4620-95c5-ce491044458c-tmp-dir\") pod \"node-resolver-krt5w\" (UID: \"a88360e6-cc58-4620-95c5-ce491044458c\") " pod="openshift-dns/node-resolver-krt5w" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261118 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-kubernetes\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261145 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-run-openvswitch\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261172 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-cni-netd\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261214 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbqxr\" (UniqueName: \"kubernetes.io/projected/0b9829e7-da95-4c05-93b8-e494a91194e9-kube-api-access-xbqxr\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261240 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-socket-dir\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261243 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-system-cni-dir\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261283 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-cnibin\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.261634 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261292 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-system-cni-dir\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261309 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-etc-openvswitch\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261337 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-ovn-node-metrics-cert\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261365 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx9rf\" (UniqueName: \"kubernetes.io/projected/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-kube-api-access-tx9rf\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261375 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c9ad3582-7d79-470d-a2e0-cfcb46eef262-serviceca\") pod \"node-ca-42h29\" (UID: \"c9ad3582-7d79-470d-a2e0-cfcb46eef262\") " pod="openshift-image-registry/node-ca-42h29" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261405 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b05d4e0-52bc-44b6-9409-f2597e4b9fd7-host-slash\") pod \"iptables-alerter-j7sfm\" (UID: \"2b05d4e0-52bc-44b6-9409-f2597e4b9fd7\") " pod="openshift-network-operator/iptables-alerter-j7sfm" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261434 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-sysctl-d\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261446 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a91426dc-0372-43c1-9929-339429bd1d6a-konnectivity-ca\") pod \"konnectivity-agent-6jbsz\" (UID: \"a91426dc-0372-43c1-9929-339429bd1d6a\") " pod="kube-system/konnectivity-agent-6jbsz" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261459 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-run-multus-certs\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261477 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c9ad3582-7d79-470d-a2e0-cfcb46eef262-host\") pod \"node-ca-42h29\" (UID: \"c9ad3582-7d79-470d-a2e0-cfcb46eef262\") " pod="openshift-image-registry/node-ca-42h29" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261484 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-cnibin\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261511 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-run\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261575 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5662m\" (UniqueName: \"kubernetes.io/projected/c9ad3582-7d79-470d-a2e0-cfcb46eef262-kube-api-access-5662m\") pod \"node-ca-42h29\" (UID: \"c9ad3582-7d79-470d-a2e0-cfcb46eef262\") " pod="openshift-image-registry/node-ca-42h29" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261607 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261616 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c9ad3582-7d79-470d-a2e0-cfcb46eef262-host\") pod \"node-ca-42h29\" (UID: \"c9ad3582-7d79-470d-a2e0-cfcb46eef262\") " pod="openshift-image-registry/node-ca-42h29" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261641 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a88360e6-cc58-4620-95c5-ce491044458c-tmp-dir\") pod \"node-resolver-krt5w\" (UID: \"a88360e6-cc58-4620-95c5-ce491044458c\") " pod="openshift-dns/node-resolver-krt5w" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261711 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-multus-socket-dir-parent\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261746 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-var-lib-cni-bin\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.262626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261798 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-var-lib-cni-bin\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261850 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-log-socket\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261873 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-multus-socket-dir-parent\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261881 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a91426dc-0372-43c1-9929-339429bd1d6a-agent-certs\") pod \"konnectivity-agent-6jbsz\" (UID: \"a91426dc-0372-43c1-9929-339429bd1d6a\") " pod="kube-system/konnectivity-agent-6jbsz" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.261911 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-sysconfig\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262032 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-var-lib-kubelet\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262038 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-sysconfig\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262091 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-os-release\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262224 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-multus-cni-dir\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262245 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-os-release\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262262 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ae642324-8238-4122-8aa2-055642caafdd-cni-binary-copy\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262295 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a91426dc-0372-43c1-9929-339429bd1d6a-konnectivity-ca\") pod \"konnectivity-agent-6jbsz\" (UID: \"a91426dc-0372-43c1-9929-339429bd1d6a\") " pod="kube-system/konnectivity-agent-6jbsz" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262265 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262299 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs\") pod \"network-metrics-daemon-d6kvg\" (UID: \"8892e9d1-ce2b-47d1-af07-27d1e94dd31c\") " pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262291 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-var-lib-kubelet\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262375 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-kubelet\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262410 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-multus-cni-dir\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.263462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262406 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262453 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9l7v\" (UniqueName: \"kubernetes.io/projected/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-kube-api-access-n9l7v\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262483 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-systemd\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262512 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262546 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-device-dir\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262509 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262581 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-systemd\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262582 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-sys\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262626 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-host\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262637 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-sys\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262638 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-device-dir\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262652 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a88360e6-cc58-4620-95c5-ce491044458c-hosts-file\") pod \"node-resolver-krt5w\" (UID: \"a88360e6-cc58-4620-95c5-ce491044458c\") " pod="openshift-dns/node-resolver-krt5w" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262687 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-run-k8s-cni-cncf-io\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262699 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-host\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262700 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a88360e6-cc58-4620-95c5-ce491044458c-hosts-file\") pod \"node-resolver-krt5w\" (UID: \"a88360e6-cc58-4620-95c5-ce491044458c\") " pod="openshift-dns/node-resolver-krt5w" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262714 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-var-lib-cni-multus\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262730 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-run-k8s-cni-cncf-io\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262742 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-hostroot\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.264338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262766 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-host-var-lib-cni-multus\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262770 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262798 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-system-cni-dir\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262804 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-hostroot\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262829 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ae642324-8238-4122-8aa2-055642caafdd-multus-daemon-config\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262833 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ae642324-8238-4122-8aa2-055642caafdd-cni-binary-copy\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262855 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-system-cni-dir\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262858 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-run-systemd\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262891 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b9829e7-da95-4c05-93b8-e494a91194e9-tmp\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.262985 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-etc-selinux\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263129 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-lib-modules\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263170 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-etc-selinux\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263178 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-node-log\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263212 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-modprobe-d\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263362 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-lib-modules\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263390 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-modprobe-d\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263427 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.265103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263468 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.265984 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263484 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k964n\" (UniqueName: \"kubernetes.io/projected/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-kube-api-access-k964n\") pod \"network-metrics-daemon-d6kvg\" (UID: \"8892e9d1-ce2b-47d1-af07-27d1e94dd31c\") " pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:09.265984 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263495 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ae642324-8238-4122-8aa2-055642caafdd-multus-daemon-config\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.265984 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263566 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-slash\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.265984 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263607 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.265984 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263633 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-ovnkube-config\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.265984 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263633 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.265984 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263663 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f0a26b92-3040-40c7-904f-7c8112a1f5e5-kubelet-config\") pod \"global-pull-secret-syncer-78h5z\" (UID: \"f0a26b92-3040-40c7-904f-7c8112a1f5e5\") " pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:09.265984 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263688 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-cnibin\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.265984 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263709 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f0a26b92-3040-40c7-904f-7c8112a1f5e5-kubelet-config\") pod \"global-pull-secret-syncer-78h5z\" (UID: \"f0a26b92-3040-40c7-904f-7c8112a1f5e5\") " pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:09.265984 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263722 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkwvq\" (UniqueName: \"kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq\") pod \"network-check-target-sjhtn\" (UID: \"cf05bfdb-d809-4562-b39f-ba0dbddd0296\") " pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:09.265984 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263757 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-cnibin\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.265984 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263776 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f0a26b92-3040-40c7-904f-7c8112a1f5e5-dbus\") pod \"global-pull-secret-syncer-78h5z\" (UID: \"f0a26b92-3040-40c7-904f-7c8112a1f5e5\") " pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:09.265984 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263835 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f0a26b92-3040-40c7-904f-7c8112a1f5e5-dbus\") pod \"global-pull-secret-syncer-78h5z\" (UID: \"f0a26b92-3040-40c7-904f-7c8112a1f5e5\") " pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:09.265984 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263832 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwsm4\" (UniqueName: \"kubernetes.io/projected/a88360e6-cc58-4620-95c5-ce491044458c-kube-api-access-vwsm4\") pod \"node-resolver-krt5w\" (UID: \"a88360e6-cc58-4620-95c5-ce491044458c\") " pod="openshift-dns/node-resolver-krt5w" Jul 16 12:39:09.265984 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263884 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-var-lib-openvswitch\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.265984 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263912 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-ovnkube-script-lib\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.266776 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263949 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret\") pod \"global-pull-secret-syncer-78h5z\" (UID: \"f0a26b92-3040-40c7-904f-7c8112a1f5e5\") " pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:09.266776 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.263976 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-registration-dir\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.266776 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.264009 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-sysctl-conf\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.266776 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.264034 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.266776 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.264061 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-os-release\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.266776 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.264156 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:39:09.266776 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.264257 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ae642324-8238-4122-8aa2-055642caafdd-os-release\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.266776 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.264318 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret podName:f0a26b92-3040-40c7-904f-7c8112a1f5e5 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:09.764295498 +0000 UTC m=+2.998203343 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret") pod "global-pull-secret-syncer-78h5z" (UID: "f0a26b92-3040-40c7-904f-7c8112a1f5e5") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:39:09.266776 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.264315 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-registration-dir\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.266776 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.264412 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-sysctl-conf\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.266776 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.264876 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.266776 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.265389 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0b9829e7-da95-4c05-93b8-e494a91194e9-etc-tuned\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.266776 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.265621 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a91426dc-0372-43c1-9929-339429bd1d6a-agent-certs\") pod \"konnectivity-agent-6jbsz\" (UID: \"a91426dc-0372-43c1-9929-339429bd1d6a\") " pod="kube-system/konnectivity-agent-6jbsz" Jul 16 12:39:09.267409 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.266924 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b9829e7-da95-4c05-93b8-e494a91194e9-tmp\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.269422 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.269373 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcl7f\" (UniqueName: \"kubernetes.io/projected/5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9-kube-api-access-dcl7f\") pod \"multus-additional-cni-plugins-v4lc5\" (UID: \"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9\") " pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.269902 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.269838 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbmrj\" (UniqueName: \"kubernetes.io/projected/ae642324-8238-4122-8aa2-055642caafdd-kube-api-access-dbmrj\") pod \"multus-lph7m\" (UID: \"ae642324-8238-4122-8aa2-055642caafdd\") " pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.270618 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.270560 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbqxr\" (UniqueName: \"kubernetes.io/projected/0b9829e7-da95-4c05-93b8-e494a91194e9-kube-api-access-xbqxr\") pod \"tuned-snw24\" (UID: \"0b9829e7-da95-4c05-93b8-e494a91194e9\") " pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.271465 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.271433 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9l7v\" (UniqueName: \"kubernetes.io/projected/a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb-kube-api-access-n9l7v\") pod \"aws-ebs-csi-driver-node-czfl6\" (UID: \"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.271825 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.271803 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5662m\" (UniqueName: \"kubernetes.io/projected/c9ad3582-7d79-470d-a2e0-cfcb46eef262-kube-api-access-5662m\") pod \"node-ca-42h29\" (UID: \"c9ad3582-7d79-470d-a2e0-cfcb46eef262\") " pod="openshift-image-registry/node-ca-42h29" Jul 16 12:39:09.275564 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.275544 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwsm4\" (UniqueName: \"kubernetes.io/projected/a88360e6-cc58-4620-95c5-ce491044458c-kube-api-access-vwsm4\") pod \"node-resolver-krt5w\" (UID: \"a88360e6-cc58-4620-95c5-ce491044458c\") " pod="openshift-dns/node-resolver-krt5w" Jul 16 12:39:09.364512 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364468 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k964n\" (UniqueName: \"kubernetes.io/projected/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-kube-api-access-k964n\") pod \"network-metrics-daemon-d6kvg\" (UID: \"8892e9d1-ce2b-47d1-af07-27d1e94dd31c\") " pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:09.364710 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364523 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-slash\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.364710 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364558 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.364710 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364588 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-ovnkube-config\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.364710 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364616 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkwvq\" (UniqueName: \"kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq\") pod \"network-check-target-sjhtn\" (UID: \"cf05bfdb-d809-4562-b39f-ba0dbddd0296\") " pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:09.364710 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364648 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-var-lib-openvswitch\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.364710 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364667 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-slash\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.364710 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364678 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-ovnkube-script-lib\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.364710 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364688 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364723 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-systemd-units\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364750 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-run-ovn\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364753 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-var-lib-openvswitch\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364773 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-cni-bin\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364802 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-systemd-units\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364804 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-env-overrides\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364853 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-run-netns\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364885 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b05d4e0-52bc-44b6-9409-f2597e4b9fd7-iptables-alerter-script\") pod \"iptables-alerter-j7sfm\" (UID: \"2b05d4e0-52bc-44b6-9409-f2597e4b9fd7\") " pod="openshift-network-operator/iptables-alerter-j7sfm" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364916 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d57m9\" (UniqueName: \"kubernetes.io/projected/2b05d4e0-52bc-44b6-9409-f2597e4b9fd7-kube-api-access-d57m9\") pod \"iptables-alerter-j7sfm\" (UID: \"2b05d4e0-52bc-44b6-9409-f2597e4b9fd7\") " pod="openshift-network-operator/iptables-alerter-j7sfm" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364931 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-run-netns\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364949 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-run-openvswitch\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.364986 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-cni-netd\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365016 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-etc-openvswitch\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365041 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-ovn-node-metrics-cert\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365068 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx9rf\" (UniqueName: \"kubernetes.io/projected/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-kube-api-access-tx9rf\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365113 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b05d4e0-52bc-44b6-9409-f2597e4b9fd7-host-slash\") pod \"iptables-alerter-j7sfm\" (UID: \"2b05d4e0-52bc-44b6-9409-f2597e4b9fd7\") " pod="openshift-network-operator/iptables-alerter-j7sfm" Jul 16 12:39:09.365161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365145 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-log-socket\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365180 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs\") pod \"network-metrics-daemon-d6kvg\" (UID: \"8892e9d1-ce2b-47d1-af07-27d1e94dd31c\") " pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365207 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-kubelet\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365235 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365275 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-run-systemd\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365308 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-node-log\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365325 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-env-overrides\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365401 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-run-ovn\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365446 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-node-log\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365446 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-ovnkube-script-lib\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365457 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-ovnkube-config\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365468 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b05d4e0-52bc-44b6-9409-f2597e4b9fd7-iptables-alerter-script\") pod \"iptables-alerter-j7sfm\" (UID: \"2b05d4e0-52bc-44b6-9409-f2597e4b9fd7\") " pod="openshift-network-operator/iptables-alerter-j7sfm" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365491 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-cni-bin\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365505 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-run-openvswitch\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365516 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b05d4e0-52bc-44b6-9409-f2597e4b9fd7-host-slash\") pod \"iptables-alerter-j7sfm\" (UID: \"2b05d4e0-52bc-44b6-9409-f2597e4b9fd7\") " pod="openshift-network-operator/iptables-alerter-j7sfm" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365533 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-log-socket\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365547 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-cni-netd\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.365566 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:39:09.365918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365578 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-etc-openvswitch\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.366624 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365588 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-kubelet\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.366624 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.365622 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs podName:8892e9d1-ce2b-47d1-af07-27d1e94dd31c nodeName:}" failed. No retries permitted until 2026-07-16 12:39:09.865605461 +0000 UTC m=+3.099513295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs") pod "network-metrics-daemon-d6kvg" (UID: "8892e9d1-ce2b-47d1-af07-27d1e94dd31c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:39:09.366624 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365626 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-run-systemd\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.366624 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.365750 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.368355 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.368323 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-ovn-node-metrics-cert\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.370607 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.370542 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:39:09.370607 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.370572 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:39:09.370607 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.370588 2567 projected.go:196] Error preparing data for projected volume kube-api-access-fkwvq for pod openshift-network-diagnostics/network-check-target-sjhtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:39:09.370885 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.370664 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq podName:cf05bfdb-d809-4562-b39f-ba0dbddd0296 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:09.87064565 +0000 UTC m=+3.104553500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fkwvq" (UniqueName: "kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq") pod "network-check-target-sjhtn" (UID: "cf05bfdb-d809-4562-b39f-ba0dbddd0296") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:39:09.373562 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.373527 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k964n\" (UniqueName: \"kubernetes.io/projected/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-kube-api-access-k964n\") pod \"network-metrics-daemon-d6kvg\" (UID: \"8892e9d1-ce2b-47d1-af07-27d1e94dd31c\") " pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:09.374544 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.374523 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx9rf\" (UniqueName: \"kubernetes.io/projected/aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9-kube-api-access-tx9rf\") pod \"ovnkube-node-s729s\" (UID: \"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.374641 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.374598 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d57m9\" (UniqueName: \"kubernetes.io/projected/2b05d4e0-52bc-44b6-9409-f2597e4b9fd7-kube-api-access-d57m9\") pod \"iptables-alerter-j7sfm\" (UID: \"2b05d4e0-52bc-44b6-9409-f2597e4b9fd7\") " pod="openshift-network-operator/iptables-alerter-j7sfm" Jul 16 12:39:09.437323 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.437291 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6jbsz" Jul 16 12:39:09.447357 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.447329 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" Jul 16 12:39:09.447994 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:09.447965 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91426dc_0372_43c1_9929_339429bd1d6a.slice/crio-6fd27ca02fb5475085c056fe1457074e0f5de496e4789563c5fd8ff0c46407cd WatchSource:0}: Error finding container 6fd27ca02fb5475085c056fe1457074e0f5de496e4789563c5fd8ff0c46407cd: Status 404 returned error can't find the container with id 6fd27ca02fb5475085c056fe1457074e0f5de496e4789563c5fd8ff0c46407cd Jul 16 12:39:09.454710 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.454692 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-snw24" Jul 16 12:39:09.457685 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:09.457659 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1c0b69c_ed8d_4a38_a7e3_aeea1a0b74fb.slice/crio-604b4585586c19a4ebed3e9f54687e530487556f497362e12ae0040d20c61a3c WatchSource:0}: Error finding container 604b4585586c19a4ebed3e9f54687e530487556f497362e12ae0040d20c61a3c: Status 404 returned error can't find the container with id 604b4585586c19a4ebed3e9f54687e530487556f497362e12ae0040d20c61a3c Jul 16 12:39:09.462049 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.462030 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-42h29" Jul 16 12:39:09.462871 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:09.462846 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b9829e7_da95_4c05_93b8_e494a91194e9.slice/crio-4f12e26a5e139d2d52c7906d1eb220b0c975148879888651e3cadb1010d7f251 WatchSource:0}: Error finding container 4f12e26a5e139d2d52c7906d1eb220b0c975148879888651e3cadb1010d7f251: Status 404 returned error can't find the container with id 4f12e26a5e139d2d52c7906d1eb220b0c975148879888651e3cadb1010d7f251 Jul 16 12:39:09.468959 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.468937 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v4lc5" Jul 16 12:39:09.473781 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:09.473761 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9ad3582_7d79_470d_a2e0_cfcb46eef262.slice/crio-933f2ee04239fac118839eb612b1faaab57cb8fb4fe4e378338c64676c766c19 WatchSource:0}: Error finding container 933f2ee04239fac118839eb612b1faaab57cb8fb4fe4e378338c64676c766c19: Status 404 returned error can't find the container with id 933f2ee04239fac118839eb612b1faaab57cb8fb4fe4e378338c64676c766c19 Jul 16 12:39:09.475769 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.475575 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-krt5w" Jul 16 12:39:09.478604 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:09.478577 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d1ebc5e_2b7a_4e52_a17b_99c9c66ff5a9.slice/crio-0331a812949f3c631fc1092259e6800e1304f308d5f6ae66ab7ccfdf4274579b WatchSource:0}: Error finding container 0331a812949f3c631fc1092259e6800e1304f308d5f6ae66ab7ccfdf4274579b: Status 404 returned error can't find the container with id 0331a812949f3c631fc1092259e6800e1304f308d5f6ae66ab7ccfdf4274579b Jul 16 12:39:09.483025 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.483006 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lph7m" Jul 16 12:39:09.485523 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:09.485495 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda88360e6_cc58_4620_95c5_ce491044458c.slice/crio-81041093fbfb451b3012f4f2bb34e2d8133c9729e25006173de883e2a47b0b2c WatchSource:0}: Error finding container 81041093fbfb451b3012f4f2bb34e2d8133c9729e25006173de883e2a47b0b2c: Status 404 returned error can't find the container with id 81041093fbfb451b3012f4f2bb34e2d8133c9729e25006173de883e2a47b0b2c Jul 16 12:39:09.491331 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.491161 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j7sfm" Jul 16 12:39:09.492530 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:09.492507 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae642324_8238_4122_8aa2_055642caafdd.slice/crio-c8c9e7026f5c17f09f12338ccfa252261e6d14ce479c8ebb3bb706828147af7e WatchSource:0}: Error finding container c8c9e7026f5c17f09f12338ccfa252261e6d14ce479c8ebb3bb706828147af7e: Status 404 returned error can't find the container with id c8c9e7026f5c17f09f12338ccfa252261e6d14ce479c8ebb3bb706828147af7e Jul 16 12:39:09.497282 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.497265 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:09.500714 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:09.500689 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b05d4e0_52bc_44b6_9409_f2597e4b9fd7.slice/crio-32bc2badaae4c702e7978e645a31ff5e3aa4f15192378c0a7284195e7b698aba WatchSource:0}: Error finding container 32bc2badaae4c702e7978e645a31ff5e3aa4f15192378c0a7284195e7b698aba: Status 404 returned error can't find the container with id 32bc2badaae4c702e7978e645a31ff5e3aa4f15192378c0a7284195e7b698aba Jul 16 12:39:09.506205 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:09.506185 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa1afabc_1ae8_4482_b6b4_b6ae5cdf88d9.slice/crio-05aa82930ecde2bbf0c63111a6334d85d2ec006b716384b3ce00b28003d6fd8d WatchSource:0}: Error finding container 05aa82930ecde2bbf0c63111a6334d85d2ec006b716384b3ce00b28003d6fd8d: Status 404 returned error can't find the container with id 05aa82930ecde2bbf0c63111a6334d85d2ec006b716384b3ce00b28003d6fd8d Jul 16 12:39:09.639125 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.639083 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" event={"ID":"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9","Type":"ContainerStarted","Data":"05aa82930ecde2bbf0c63111a6334d85d2ec006b716384b3ce00b28003d6fd8d"} Jul 16 12:39:09.640342 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.640308 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j7sfm" event={"ID":"2b05d4e0-52bc-44b6-9409-f2597e4b9fd7","Type":"ContainerStarted","Data":"32bc2badaae4c702e7978e645a31ff5e3aa4f15192378c0a7284195e7b698aba"} Jul 16 12:39:09.641665 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.641634 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lph7m" event={"ID":"ae642324-8238-4122-8aa2-055642caafdd","Type":"ContainerStarted","Data":"c8c9e7026f5c17f09f12338ccfa252261e6d14ce479c8ebb3bb706828147af7e"} Jul 16 12:39:09.642794 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.642769 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-snw24" event={"ID":"0b9829e7-da95-4c05-93b8-e494a91194e9","Type":"ContainerStarted","Data":"4f12e26a5e139d2d52c7906d1eb220b0c975148879888651e3cadb1010d7f251"} Jul 16 12:39:09.643915 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.643891 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6jbsz" event={"ID":"a91426dc-0372-43c1-9929-339429bd1d6a","Type":"ContainerStarted","Data":"6fd27ca02fb5475085c056fe1457074e0f5de496e4789563c5fd8ff0c46407cd"} Jul 16 12:39:09.646724 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.645817 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-krt5w" event={"ID":"a88360e6-cc58-4620-95c5-ce491044458c","Type":"ContainerStarted","Data":"81041093fbfb451b3012f4f2bb34e2d8133c9729e25006173de883e2a47b0b2c"} Jul 16 12:39:09.648468 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.648445 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4lc5" event={"ID":"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9","Type":"ContainerStarted","Data":"0331a812949f3c631fc1092259e6800e1304f308d5f6ae66ab7ccfdf4274579b"} Jul 16 12:39:09.649734 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.649708 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-42h29" event={"ID":"c9ad3582-7d79-470d-a2e0-cfcb46eef262","Type":"ContainerStarted","Data":"933f2ee04239fac118839eb612b1faaab57cb8fb4fe4e378338c64676c766c19"} Jul 16 12:39:09.650924 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.650903 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" event={"ID":"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb","Type":"ContainerStarted","Data":"604b4585586c19a4ebed3e9f54687e530487556f497362e12ae0040d20c61a3c"} Jul 16 12:39:09.768265 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.768180 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret\") pod \"global-pull-secret-syncer-78h5z\" (UID: \"f0a26b92-3040-40c7-904f-7c8112a1f5e5\") " pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:09.768429 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.768347 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:39:09.768429 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.768416 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret podName:f0a26b92-3040-40c7-904f-7c8112a1f5e5 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:10.768396226 +0000 UTC m=+4.002304071 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret") pod "global-pull-secret-syncer-78h5z" (UID: "f0a26b92-3040-40c7-904f-7c8112a1f5e5") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:39:09.868815 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.868782 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs\") pod \"network-metrics-daemon-d6kvg\" (UID: \"8892e9d1-ce2b-47d1-af07-27d1e94dd31c\") " pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:09.868995 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.868901 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:39:09.868995 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.868960 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs podName:8892e9d1-ce2b-47d1-af07-27d1e94dd31c nodeName:}" failed. No retries permitted until 2026-07-16 12:39:10.868945384 +0000 UTC m=+4.102853233 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs") pod "network-metrics-daemon-d6kvg" (UID: "8892e9d1-ce2b-47d1-af07-27d1e94dd31c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:39:09.969304 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:09.969271 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkwvq\" (UniqueName: \"kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq\") pod \"network-check-target-sjhtn\" (UID: \"cf05bfdb-d809-4562-b39f-ba0dbddd0296\") " pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:09.969448 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.969437 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:39:09.969485 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.969460 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:39:09.969485 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.969474 2567 projected.go:196] Error preparing data for projected volume kube-api-access-fkwvq for pod openshift-network-diagnostics/network-check-target-sjhtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:39:09.969569 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:09.969542 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq podName:cf05bfdb-d809-4562-b39f-ba0dbddd0296 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:10.969522495 +0000 UTC m=+4.203430328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fkwvq" (UniqueName: "kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq") pod "network-check-target-sjhtn" (UID: "cf05bfdb-d809-4562-b39f-ba0dbddd0296") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:39:10.183620 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:10.183546 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 12:34:09 +0000 UTC" deadline="2028-02-29 06:24:58.036895421 +0000 UTC" Jul 16 12:39:10.183620 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:10.183611 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14225h45m47.853288865s" Jul 16 12:39:10.497845 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:10.496023 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33fcd2db8501242e604d3c0ecb5673f6.slice/crio-ddd03444ee3929ded107ae308713a879a1797d6be74ca7cc2783df8c94c51a27 WatchSource:0}: Error finding container ddd03444ee3929ded107ae308713a879a1797d6be74ca7cc2783df8c94c51a27: Status 404 returned error can't find the container with id ddd03444ee3929ded107ae308713a879a1797d6be74ca7cc2783df8c94c51a27 Jul 16 12:39:10.633994 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:10.633053 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:10.633994 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:10.633237 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:10.633994 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:10.633687 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:10.633994 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:10.633791 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:10.633994 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:10.633861 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:10.633994 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:10.633945 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:10.661661 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:10.660963 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal" event={"ID":"acfaee3e8f27ae7b52fdbaeafbb5fcbe","Type":"ContainerStarted","Data":"e5dc86d57f1dc4e39f66aeb631f22adeea633aeb85692e8267788cc9e2f6da48"} Jul 16 12:39:10.663631 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:10.663558 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" event={"ID":"33fcd2db8501242e604d3c0ecb5673f6","Type":"ContainerStarted","Data":"ddd03444ee3929ded107ae308713a879a1797d6be74ca7cc2783df8c94c51a27"} Jul 16 12:39:10.779225 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:10.778824 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret\") pod \"global-pull-secret-syncer-78h5z\" (UID: \"f0a26b92-3040-40c7-904f-7c8112a1f5e5\") " pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:10.779225 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:10.779064 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:39:10.779936 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:10.779903 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret podName:f0a26b92-3040-40c7-904f-7c8112a1f5e5 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:12.779111844 +0000 UTC m=+6.013019679 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret") pod "global-pull-secret-syncer-78h5z" (UID: "f0a26b92-3040-40c7-904f-7c8112a1f5e5") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:39:10.880417 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:10.879649 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs\") pod \"network-metrics-daemon-d6kvg\" (UID: \"8892e9d1-ce2b-47d1-af07-27d1e94dd31c\") " pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:10.880417 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:10.879889 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:39:10.880417 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:10.879967 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs podName:8892e9d1-ce2b-47d1-af07-27d1e94dd31c nodeName:}" failed. No retries permitted until 2026-07-16 12:39:12.879943881 +0000 UTC m=+6.113851712 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs") pod "network-metrics-daemon-d6kvg" (UID: "8892e9d1-ce2b-47d1-af07-27d1e94dd31c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:39:10.980737 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:10.980453 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkwvq\" (UniqueName: \"kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq\") pod \"network-check-target-sjhtn\" (UID: \"cf05bfdb-d809-4562-b39f-ba0dbddd0296\") " pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:10.980944 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:10.980758 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:39:10.980944 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:10.980787 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:39:10.980944 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:10.980803 2567 projected.go:196] Error preparing data for projected volume kube-api-access-fkwvq for pod openshift-network-diagnostics/network-check-target-sjhtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:39:10.980944 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:10.980880 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq podName:cf05bfdb-d809-4562-b39f-ba0dbddd0296 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:12.980857877 +0000 UTC m=+6.214765711 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fkwvq" (UniqueName: "kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq") pod "network-check-target-sjhtn" (UID: "cf05bfdb-d809-4562-b39f-ba0dbddd0296") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:39:11.184812 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:11.184738 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 12:34:09 +0000 UTC" deadline="2028-02-18 14:33:15.451752281 +0000 UTC" Jul 16 12:39:11.184812 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:11.184809 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13969h54m4.266946917s" Jul 16 12:39:12.633463 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:12.632989 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:12.633463 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:12.633149 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:12.633463 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:12.633225 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:12.633463 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:12.633140 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:12.633463 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:12.633293 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:12.633463 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:12.633379 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:12.799592 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:12.799552 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret\") pod \"global-pull-secret-syncer-78h5z\" (UID: \"f0a26b92-3040-40c7-904f-7c8112a1f5e5\") " pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:12.799756 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:12.799732 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:39:12.799813 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:12.799794 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret podName:f0a26b92-3040-40c7-904f-7c8112a1f5e5 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:16.799776378 +0000 UTC m=+10.033684209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret") pod "global-pull-secret-syncer-78h5z" (UID: "f0a26b92-3040-40c7-904f-7c8112a1f5e5") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:39:12.900446 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:12.900353 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs\") pod \"network-metrics-daemon-d6kvg\" (UID: \"8892e9d1-ce2b-47d1-af07-27d1e94dd31c\") " pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:12.900618 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:12.900557 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:39:12.900680 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:12.900622 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs podName:8892e9d1-ce2b-47d1-af07-27d1e94dd31c nodeName:}" failed. No retries permitted until 2026-07-16 12:39:16.900603253 +0000 UTC m=+10.134511090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs") pod "network-metrics-daemon-d6kvg" (UID: "8892e9d1-ce2b-47d1-af07-27d1e94dd31c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:39:13.001598 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:13.000931 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkwvq\" (UniqueName: \"kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq\") pod \"network-check-target-sjhtn\" (UID: \"cf05bfdb-d809-4562-b39f-ba0dbddd0296\") " pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:13.001598 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:13.001162 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:39:13.001598 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:13.001181 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:39:13.001598 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:13.001194 2567 projected.go:196] Error preparing data for projected volume kube-api-access-fkwvq for pod openshift-network-diagnostics/network-check-target-sjhtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:39:13.001598 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:13.001252 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq podName:cf05bfdb-d809-4562-b39f-ba0dbddd0296 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:17.001233418 +0000 UTC m=+10.235141263 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fkwvq" (UniqueName: "kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq") pod "network-check-target-sjhtn" (UID: "cf05bfdb-d809-4562-b39f-ba0dbddd0296") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:39:14.633276 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:14.633218 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:14.633276 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:14.633243 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:14.633276 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:14.633243 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:14.633850 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:14.633372 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:14.633850 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:14.633457 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:14.633970 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:14.633849 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:16.633494 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:16.633429 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:16.634136 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:16.633439 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:16.634136 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:16.633585 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:16.634136 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:16.633574 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:16.634136 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:16.633693 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:16.634136 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:16.633771 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:16.837351 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:16.837309 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret\") pod \"global-pull-secret-syncer-78h5z\" (UID: \"f0a26b92-3040-40c7-904f-7c8112a1f5e5\") " pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:16.837554 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:16.837435 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:39:16.837554 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:16.837492 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret podName:f0a26b92-3040-40c7-904f-7c8112a1f5e5 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:24.837474474 +0000 UTC m=+18.071382302 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret") pod "global-pull-secret-syncer-78h5z" (UID: "f0a26b92-3040-40c7-904f-7c8112a1f5e5") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:39:16.938348 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:16.938261 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs\") pod \"network-metrics-daemon-d6kvg\" (UID: \"8892e9d1-ce2b-47d1-af07-27d1e94dd31c\") " pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:16.938522 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:16.938433 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:39:16.938522 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:16.938517 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs podName:8892e9d1-ce2b-47d1-af07-27d1e94dd31c nodeName:}" failed. No retries permitted until 2026-07-16 12:39:24.93849497 +0000 UTC m=+18.172402805 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs") pod "network-metrics-daemon-d6kvg" (UID: "8892e9d1-ce2b-47d1-af07-27d1e94dd31c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:39:17.039281 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:17.039222 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkwvq\" (UniqueName: \"kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq\") pod \"network-check-target-sjhtn\" (UID: \"cf05bfdb-d809-4562-b39f-ba0dbddd0296\") " pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:17.039477 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:17.039398 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:39:17.039477 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:17.039426 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:39:17.039477 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:17.039441 2567 projected.go:196] Error preparing data for projected volume kube-api-access-fkwvq for pod openshift-network-diagnostics/network-check-target-sjhtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:39:17.039629 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:17.039497 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq podName:cf05bfdb-d809-4562-b39f-ba0dbddd0296 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:25.039478633 +0000 UTC m=+18.273386461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fkwvq" (UniqueName: "kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq") pod "network-check-target-sjhtn" (UID: "cf05bfdb-d809-4562-b39f-ba0dbddd0296") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:39:18.633463 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:18.633383 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:18.633463 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:18.633427 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:18.633997 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:18.633390 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:18.633997 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:18.633542 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:18.633997 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:18.633668 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:18.633997 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:18.633760 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:20.633532 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:20.633491 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:20.633532 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:20.633517 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:20.634018 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:20.633508 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:20.634018 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:20.633672 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:20.634018 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:20.633815 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:20.634018 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:20.633899 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:22.633359 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:22.633328 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:22.633815 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:22.633442 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:22.633815 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:22.633329 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:22.633815 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:22.633329 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:22.633815 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:22.633535 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:22.633815 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:22.633580 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:24.633973 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:24.633935 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:24.633973 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:24.633979 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:24.634561 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:24.634086 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:24.634561 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:24.634146 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:24.634561 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:24.634257 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:24.634561 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:24.634337 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:24.897664 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:24.897590 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret\") pod \"global-pull-secret-syncer-78h5z\" (UID: \"f0a26b92-3040-40c7-904f-7c8112a1f5e5\") " pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:24.897815 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:24.897705 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:39:24.897815 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:24.897784 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret podName:f0a26b92-3040-40c7-904f-7c8112a1f5e5 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:40.897759679 +0000 UTC m=+34.131667512 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret") pod "global-pull-secret-syncer-78h5z" (UID: "f0a26b92-3040-40c7-904f-7c8112a1f5e5") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:39:24.998150 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:24.998120 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs\") pod \"network-metrics-daemon-d6kvg\" (UID: \"8892e9d1-ce2b-47d1-af07-27d1e94dd31c\") " pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:24.998297 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:24.998227 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:39:24.998297 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:24.998277 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs podName:8892e9d1-ce2b-47d1-af07-27d1e94dd31c nodeName:}" failed. No retries permitted until 2026-07-16 12:39:40.998264729 +0000 UTC m=+34.232172559 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs") pod "network-metrics-daemon-d6kvg" (UID: "8892e9d1-ce2b-47d1-af07-27d1e94dd31c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:39:25.098650 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:25.098617 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkwvq\" (UniqueName: \"kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq\") pod \"network-check-target-sjhtn\" (UID: \"cf05bfdb-d809-4562-b39f-ba0dbddd0296\") " pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:25.098825 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:25.098805 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:39:25.098870 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:25.098834 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:39:25.098870 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:25.098847 2567 projected.go:196] Error preparing data for projected volume kube-api-access-fkwvq for pod openshift-network-diagnostics/network-check-target-sjhtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:39:25.098961 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:25.098907 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq podName:cf05bfdb-d809-4562-b39f-ba0dbddd0296 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:41.098891152 +0000 UTC m=+34.332798994 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fkwvq" (UniqueName: "kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq") pod "network-check-target-sjhtn" (UID: "cf05bfdb-d809-4562-b39f-ba0dbddd0296") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:39:26.634003 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:26.633948 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:26.634438 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:26.634013 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:26.634438 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:26.633962 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:26.634438 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:26.634118 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:26.634438 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:26.634196 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:26.634438 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:26.634296 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:28.632950 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:28.632913 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:28.633423 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:28.632959 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:28.633423 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:28.633026 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:28.633423 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:28.633037 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:28.633423 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:28.633128 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:28.633423 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:28.633219 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:29.713986 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.713740 2567 generic.go:356] "Generic (PLEG): container finished" podID="33fcd2db8501242e604d3c0ecb5673f6" containerID="1a0ccee61628cb4b30fad3cff9bebb6fc287949cd4950d5b6479fe3cef9b9400" exitCode=0 Jul 16 12:39:29.714636 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.713827 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" event={"ID":"33fcd2db8501242e604d3c0ecb5673f6","Type":"ContainerDied","Data":"1a0ccee61628cb4b30fad3cff9bebb6fc287949cd4950d5b6479fe3cef9b9400"} Jul 16 12:39:29.715835 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.715816 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 12:39:29.716216 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.716197 2567 generic.go:356] "Generic (PLEG): container finished" podID="aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9" containerID="25444a1f749e9caa7906de9c93d0a4d397a50177fcb97eb647b752e9e8dc44ef" exitCode=1 Jul 16 12:39:29.716312 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.716263 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" event={"ID":"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9","Type":"ContainerStarted","Data":"dc08e2ba1465afa3952dda4e2c8bf185326f38e0fbf84178f1c1c709643288e0"} Jul 16 12:39:29.716368 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.716353 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" event={"ID":"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9","Type":"ContainerDied","Data":"25444a1f749e9caa7906de9c93d0a4d397a50177fcb97eb647b752e9e8dc44ef"} Jul 16 12:39:29.716427 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.716413 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" event={"ID":"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9","Type":"ContainerStarted","Data":"dbe8ea1ec114a1b9ecd79b9ba76fe060380ef76006b5829e3b1c6ee95e95ff11"} Jul 16 12:39:29.717646 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.717620 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lph7m" event={"ID":"ae642324-8238-4122-8aa2-055642caafdd","Type":"ContainerStarted","Data":"46d01cc694eb7876ccce245bb984933dd86d0f2b04cae270fd324699946deed8"} Jul 16 12:39:29.719084 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.719039 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-snw24" event={"ID":"0b9829e7-da95-4c05-93b8-e494a91194e9","Type":"ContainerStarted","Data":"35f79d0e5a9dfe56815d9ae02c1dbbcf193ace731569691fb5f1ad9df0117905"} Jul 16 12:39:29.720222 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.720206 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6jbsz" event={"ID":"a91426dc-0372-43c1-9929-339429bd1d6a","Type":"ContainerStarted","Data":"f72947388d1fbab4184b9a5831b8b435565e7382b1b7501373a31a7dab9bcf2f"} Jul 16 12:39:29.721377 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.721356 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-krt5w" event={"ID":"a88360e6-cc58-4620-95c5-ce491044458c","Type":"ContainerStarted","Data":"a0fdf9e415ee43b993847c392b4b313170b92a8b6cd9591f225b1aadc62e2448"} Jul 16 12:39:29.722717 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.722696 2567 generic.go:356] "Generic (PLEG): container finished" podID="5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9" containerID="46dca4e9c66ea02ec5cef783b0357f5a0b090d33c63be6dfad2309671617a972" exitCode=0 Jul 16 12:39:29.722798 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.722759 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4lc5" event={"ID":"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9","Type":"ContainerDied","Data":"46dca4e9c66ea02ec5cef783b0357f5a0b090d33c63be6dfad2309671617a972"} Jul 16 12:39:29.724234 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.724206 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-42h29" event={"ID":"c9ad3582-7d79-470d-a2e0-cfcb46eef262","Type":"ContainerStarted","Data":"52763e4146becff1559fdced7b47c48cbf62dc55ec810816786f1b3531718b88"} Jul 16 12:39:29.725607 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.725553 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" event={"ID":"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb","Type":"ContainerStarted","Data":"243783fc3bc6ee3e3eb6db95e0f86f39a0d15ddb186f9c0c44fd5268b0c63dfd"} Jul 16 12:39:29.729940 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.729893 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal" podStartSLOduration=21.729876226000002 podStartE2EDuration="21.729876226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:39:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:39:10.68203639 +0000 UTC m=+3.915944256" watchObservedRunningTime="2026-07-16 12:39:29.729876226 +0000 UTC m=+22.963784077" Jul 16 12:39:29.746775 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.746725 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-snw24" podStartSLOduration=1.979380557 podStartE2EDuration="21.746705186s" totalImagesPullingTime="19.767324629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:39:29.746491281 +0000 UTC m=+22.980399132" watchObservedRunningTime="2026-07-16 12:39:29.746705186 +0000 UTC m=+22.980613038" Jul 16 12:39:29.764839 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.764792 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-krt5w" podStartSLOduration=2.041003658 podStartE2EDuration="21.76477585s" totalImagesPullingTime="19.723772192s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:39:29.764590198 +0000 UTC m=+22.998498049" watchObservedRunningTime="2026-07-16 12:39:29.76477585 +0000 UTC m=+22.998683703" Jul 16 12:39:29.791136 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.791093 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lph7m" podStartSLOduration=2.018445574 podStartE2EDuration="21.791079738s" totalImagesPullingTime="19.772634164s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:39:29.786211702 +0000 UTC m=+23.020119568" watchObservedRunningTime="2026-07-16 12:39:29.791079738 +0000 UTC m=+23.024987586" Jul 16 12:39:29.801377 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.801341 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6jbsz" podStartSLOduration=2.037701782 podStartE2EDuration="21.801330513s" totalImagesPullingTime="19.763628731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:39:29.801012367 +0000 UTC m=+23.034920219" watchObservedRunningTime="2026-07-16 12:39:29.801330513 +0000 UTC m=+23.035238415" Jul 16 12:39:29.852516 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:29.852471 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-42h29" podStartSLOduration=2.115907173 podStartE2EDuration="21.852454815s" totalImagesPullingTime="19.736547642s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:39:29.825746591 +0000 UTC m=+23.059654441" watchObservedRunningTime="2026-07-16 12:39:29.852454815 +0000 UTC m=+23.086362666" Jul 16 12:39:30.260548 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:30.260513 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6jbsz" Jul 16 12:39:30.261605 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:30.261584 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6jbsz" Jul 16 12:39:30.633300 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:30.633270 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:30.633468 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:30.633318 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:30.633468 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:30.633385 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:30.633468 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:30.633398 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:30.633641 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:30.633464 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:30.633641 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:30.633578 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:30.731322 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:30.731294 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 12:39:30.731788 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:30.731749 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" event={"ID":"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9","Type":"ContainerStarted","Data":"7d5cc775ec10b88882064c031ae3f165f689d340d75b7b60dfbe38cee7ee8c81"} Jul 16 12:39:30.731849 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:30.731838 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" event={"ID":"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9","Type":"ContainerStarted","Data":"b7b10bd2b88817c433fe8aa2e59dc40673890fc734de6440b133cbeca47e058d"} Jul 16 12:39:30.731903 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:30.731892 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" event={"ID":"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9","Type":"ContainerStarted","Data":"7d9376c1e4d85e3d4cef8babe4aba57f710708b7c12bd66ccc73db165dd7d9dc"} Jul 16 12:39:30.733424 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:30.733391 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j7sfm" event={"ID":"2b05d4e0-52bc-44b6-9409-f2597e4b9fd7","Type":"ContainerStarted","Data":"fef2b6c8705d725693bb10ab8021e43083a737491045e0779799df063be8d456"} Jul 16 12:39:30.735603 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:30.735445 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" event={"ID":"33fcd2db8501242e604d3c0ecb5673f6","Type":"ContainerStarted","Data":"197118fcec6c9410312e79c85d0712b11e99044b07c3ebac7a62e5b93ac38481"} Jul 16 12:39:30.736280 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:30.736247 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6jbsz" Jul 16 12:39:30.736405 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:30.736313 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6jbsz" Jul 16 12:39:30.752027 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:30.751984 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-j7sfm" podStartSLOduration=3.042753105 podStartE2EDuration="22.751969912s" totalImagesPullingTime="19.709216807s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:39:30.75148415 +0000 UTC m=+23.985392000" watchObservedRunningTime="2026-07-16 12:39:30.751969912 +0000 UTC m=+23.985877763" Jul 16 12:39:30.794227 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:30.794184 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" podStartSLOduration=22.794169341 podStartE2EDuration="22.794169341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:39:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:39:30.793660595 +0000 UTC m=+24.027568446" watchObservedRunningTime="2026-07-16 12:39:30.794169341 +0000 UTC m=+24.028077192" Jul 16 12:39:32.633747 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:32.633713 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:32.634196 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:32.633755 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:32.634196 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:32.633755 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:32.634196 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:32.633818 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:32.634196 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:32.633936 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:32.634196 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:32.634035 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:33.745477 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:33.745317 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 12:39:33.745871 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:33.745819 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" event={"ID":"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9","Type":"ContainerStarted","Data":"0c118777fd299f4c19e2b3024cd9671658fe43961ddf329ac8d657d0332345f2"} Jul 16 12:39:34.357852 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:34.357828 2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 12:39:34.584041 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:34.583939 2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T12:39:34.357843554Z","UUID":"7d5b231a-37e3-4e79-962a-1696e165725a","Handler":null,"Name":"","Endpoint":""} Jul 16 12:39:34.585312 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:34.585292 2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 12:39:34.585437 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:34.585328 2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 12:39:34.632954 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:34.632923 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:34.632954 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:34.632952 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:34.633157 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:34.633017 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:34.633157 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:34.633083 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:34.633157 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:34.633148 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:34.633252 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:34.633204 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:34.749554 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:34.749519 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" event={"ID":"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb","Type":"ContainerStarted","Data":"dfc79033c032250ed175b3870ba61e89fe6afc46b667ac4eb2c4c57d7a233019"} Jul 16 12:39:34.751345 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:34.751318 2567 generic.go:356] "Generic (PLEG): container finished" podID="5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9" containerID="58896c5326fc734d79036632d85245dd3874527c26ed5f1ec238fab68136bc1c" exitCode=0 Jul 16 12:39:34.751468 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:34.751359 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4lc5" event={"ID":"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9","Type":"ContainerDied","Data":"58896c5326fc734d79036632d85245dd3874527c26ed5f1ec238fab68136bc1c"} Jul 16 12:39:36.633515 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:36.633467 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:36.634026 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:36.633467 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:36.634026 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:36.633598 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:36.634026 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:36.633608 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:36.634026 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:36.633725 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:36.634026 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:36.633796 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:36.757127 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:36.757088 2567 generic.go:356] "Generic (PLEG): container finished" podID="5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9" containerID="4c6b301060fd5d573cb54f9f680ef2071c59a5d779410e61937180fb70a32d28" exitCode=0 Jul 16 12:39:36.757303 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:36.757168 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4lc5" event={"ID":"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9","Type":"ContainerDied","Data":"4c6b301060fd5d573cb54f9f680ef2071c59a5d779410e61937180fb70a32d28"} Jul 16 12:39:36.759305 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:36.759275 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" event={"ID":"a1c0b69c-ed8d-4a38-a7e3-aeea1a0b74fb","Type":"ContainerStarted","Data":"c91d12b810e433d1d147d659ed167d91a88d6680e5e1b94a60710d27c625da16"} Jul 16 12:39:36.762689 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:36.762674 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 12:39:36.763016 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:36.762999 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" event={"ID":"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9","Type":"ContainerStarted","Data":"52c9680b0f06c0adbf34a83d20dbab3c7142dc7944c49da23538e49a075e1be6"} Jul 16 12:39:36.763300 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:36.763269 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:36.763404 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:36.763325 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:36.763404 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:36.763370 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:36.763485 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:36.763401 2567 scope.go:117] "RemoveContainer" containerID="25444a1f749e9caa7906de9c93d0a4d397a50177fcb97eb647b752e9e8dc44ef" Jul 16 12:39:36.779499 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:36.779477 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:36.779610 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:36.779580 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:39:36.808688 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:36.808644 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-czfl6" podStartSLOduration=2.764088054 podStartE2EDuration="28.808626816s" totalImagesPullingTime="26.044538762s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:39:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:39:36.808400656 +0000 UTC m=+30.042308507" watchObservedRunningTime="2026-07-16 12:39:36.808626816 +0000 UTC m=+30.042534667" Jul 16 12:39:37.770874 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:37.770845 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 12:39:37.771476 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:37.771262 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" event={"ID":"aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9","Type":"ContainerStarted","Data":"b8d15e61d6b4eb03d10873ff591bbaca4f649ab1951c3eed2bf7f86ff87aa576"} Jul 16 12:39:37.801714 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:37.801663 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" podStartSLOduration=10.027216377 podStartE2EDuration="29.801646288s" totalImagesPullingTime="19.774429911s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:39:37.801499869 +0000 UTC m=+31.035407719" watchObservedRunningTime="2026-07-16 12:39:37.801646288 +0000 UTC m=+31.035554138" Jul 16 12:39:38.203203 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:38.203036 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-78h5z"] Jul 16 12:39:38.203326 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:38.203304 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:38.203434 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:38.203410 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:38.206200 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:38.206178 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d6kvg"] Jul 16 12:39:38.206316 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:38.206302 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:38.206427 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:38.206410 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:38.206683 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:38.206660 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sjhtn"] Jul 16 12:39:38.206774 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:38.206760 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:38.206872 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:38.206853 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:38.776787 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:38.776746 2567 generic.go:356] "Generic (PLEG): container finished" podID="5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9" containerID="91ac71986043ede8f1f3178c13652f3cb323045e31cc18e920bd7eb75c12094b" exitCode=0 Jul 16 12:39:38.777468 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:38.776829 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4lc5" event={"ID":"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9","Type":"ContainerDied","Data":"91ac71986043ede8f1f3178c13652f3cb323045e31cc18e920bd7eb75c12094b"} Jul 16 12:39:39.633193 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:39.633155 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:39.633362 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:39.633208 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:39.633362 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:39.633287 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:39.633362 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:39.633325 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:39.633521 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:39.633421 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:39.633521 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:39.633502 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:40.919552 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:40.919515 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret\") pod \"global-pull-secret-syncer-78h5z\" (UID: \"f0a26b92-3040-40c7-904f-7c8112a1f5e5\") " pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:40.920083 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:40.919625 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:39:40.920083 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:40.919701 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret podName:f0a26b92-3040-40c7-904f-7c8112a1f5e5 nodeName:}" failed. No retries permitted until 2026-07-16 12:40:12.919679839 +0000 UTC m=+66.153587678 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret") pod "global-pull-secret-syncer-78h5z" (UID: "f0a26b92-3040-40c7-904f-7c8112a1f5e5") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:39:41.020345 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:41.020291 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs\") pod \"network-metrics-daemon-d6kvg\" (UID: \"8892e9d1-ce2b-47d1-af07-27d1e94dd31c\") " pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:41.020529 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:41.020455 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:39:41.020596 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:41.020533 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs podName:8892e9d1-ce2b-47d1-af07-27d1e94dd31c nodeName:}" failed. No retries permitted until 2026-07-16 12:40:13.020512765 +0000 UTC m=+66.254420607 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs") pod "network-metrics-daemon-d6kvg" (UID: "8892e9d1-ce2b-47d1-af07-27d1e94dd31c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:39:41.120756 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:41.120718 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkwvq\" (UniqueName: \"kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq\") pod \"network-check-target-sjhtn\" (UID: \"cf05bfdb-d809-4562-b39f-ba0dbddd0296\") " pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:41.120920 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:41.120876 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:39:41.120920 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:41.120899 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:39:41.120920 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:41.120908 2567 projected.go:196] Error preparing data for projected volume kube-api-access-fkwvq for pod openshift-network-diagnostics/network-check-target-sjhtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:39:41.121038 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:41.120960 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq podName:cf05bfdb-d809-4562-b39f-ba0dbddd0296 nodeName:}" failed. No retries permitted until 2026-07-16 12:40:13.120946736 +0000 UTC m=+66.354854578 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fkwvq" (UniqueName: "kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq") pod "network-check-target-sjhtn" (UID: "cf05bfdb-d809-4562-b39f-ba0dbddd0296") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:39:41.633808 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:41.633769 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:41.634009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:41.633768 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:41.634009 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:41.633916 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6kvg" podUID="8892e9d1-ce2b-47d1-af07-27d1e94dd31c" Jul 16 12:39:41.634009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:41.633931 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:41.634193 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:41.634029 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78h5z" podUID="f0a26b92-3040-40c7-904f-7c8112a1f5e5" Jul 16 12:39:41.634193 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:41.634163 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sjhtn" podUID="cf05bfdb-d809-4562-b39f-ba0dbddd0296" Jul 16 12:39:42.044491 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.044457 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeReady" Jul 16 12:39:42.045062 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.044620 2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 12:39:42.090145 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.090107 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-67vll"] Jul 16 12:39:42.116333 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.116295 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-j8t4c"] Jul 16 12:39:42.116508 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.116463 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-67vll" Jul 16 12:39:42.119404 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.119379 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 12:39:42.119404 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.119393 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4kt9r\"" Jul 16 12:39:42.119619 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.119495 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 12:39:42.127481 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.127451 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84dzt\" (UniqueName: \"kubernetes.io/projected/f0f292a3-1e1b-49b2-a900-14563f2a9e15-kube-api-access-84dzt\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:39:42.127619 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.127562 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0f292a3-1e1b-49b2-a900-14563f2a9e15-tmp-dir\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:39:42.127619 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.127593 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f0f292a3-1e1b-49b2-a900-14563f2a9e15-config-volume\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:39:42.127731 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.127620 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:39:42.134269 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.134250 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-67vll"] Jul 16 12:39:42.134269 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.134282 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j8t4c"] Jul 16 12:39:42.134446 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.134364 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j8t4c" Jul 16 12:39:42.137502 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.137479 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 12:39:42.137623 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.137598 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 12:39:42.137887 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.137865 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 12:39:42.147945 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.147922 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qr4zr\"" Jul 16 12:39:42.228405 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.228365 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84dzt\" (UniqueName: \"kubernetes.io/projected/f0f292a3-1e1b-49b2-a900-14563f2a9e15-kube-api-access-84dzt\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:39:42.228606 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.228432 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0f292a3-1e1b-49b2-a900-14563f2a9e15-tmp-dir\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:39:42.228606 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.228461 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f0f292a3-1e1b-49b2-a900-14563f2a9e15-config-volume\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:39:42.228606 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.228483 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:39:42.228606 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.228565 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e2c9d30-2f89-4a05-984d-08424b633851-cert\") pod \"ingress-canary-j8t4c\" (UID: \"3e2c9d30-2f89-4a05-984d-08424b633851\") " pod="openshift-ingress-canary/ingress-canary-j8t4c" Jul 16 12:39:42.228606 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.228593 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knwc9\" (UniqueName: \"kubernetes.io/projected/3e2c9d30-2f89-4a05-984d-08424b633851-kube-api-access-knwc9\") pod \"ingress-canary-j8t4c\" (UID: \"3e2c9d30-2f89-4a05-984d-08424b633851\") " pod="openshift-ingress-canary/ingress-canary-j8t4c" Jul 16 12:39:42.228877 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:42.228696 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 12:39:42.228877 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:42.228761 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls podName:f0f292a3-1e1b-49b2-a900-14563f2a9e15 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:42.728740541 +0000 UTC m=+35.962648373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls") pod "dns-default-67vll" (UID: "f0f292a3-1e1b-49b2-a900-14563f2a9e15") : secret "dns-default-metrics-tls" not found Jul 16 12:39:42.228877 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.228821 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0f292a3-1e1b-49b2-a900-14563f2a9e15-tmp-dir\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:39:42.229159 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.229142 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f0f292a3-1e1b-49b2-a900-14563f2a9e15-config-volume\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:39:42.239453 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.239427 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84dzt\" (UniqueName: \"kubernetes.io/projected/f0f292a3-1e1b-49b2-a900-14563f2a9e15-kube-api-access-84dzt\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:39:42.329568 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.329480 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e2c9d30-2f89-4a05-984d-08424b633851-cert\") pod \"ingress-canary-j8t4c\" (UID: \"3e2c9d30-2f89-4a05-984d-08424b633851\") " pod="openshift-ingress-canary/ingress-canary-j8t4c" Jul 16 12:39:42.329568 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.329524 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knwc9\" (UniqueName: \"kubernetes.io/projected/3e2c9d30-2f89-4a05-984d-08424b633851-kube-api-access-knwc9\") pod \"ingress-canary-j8t4c\" (UID: \"3e2c9d30-2f89-4a05-984d-08424b633851\") " pod="openshift-ingress-canary/ingress-canary-j8t4c" Jul 16 12:39:42.332254 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.332224 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e2c9d30-2f89-4a05-984d-08424b633851-cert\") pod \"ingress-canary-j8t4c\" (UID: \"3e2c9d30-2f89-4a05-984d-08424b633851\") " pod="openshift-ingress-canary/ingress-canary-j8t4c" Jul 16 12:39:42.342726 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.342702 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knwc9\" (UniqueName: \"kubernetes.io/projected/3e2c9d30-2f89-4a05-984d-08424b633851-kube-api-access-knwc9\") pod \"ingress-canary-j8t4c\" (UID: \"3e2c9d30-2f89-4a05-984d-08424b633851\") " pod="openshift-ingress-canary/ingress-canary-j8t4c" Jul 16 12:39:42.445174 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.445137 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j8t4c" Jul 16 12:39:42.731948 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.731783 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:39:42.731948 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:42.731934 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 12:39:42.732153 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:42.732004 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls podName:f0f292a3-1e1b-49b2-a900-14563f2a9e15 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:43.731984651 +0000 UTC m=+36.965892489 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls") pod "dns-default-67vll" (UID: "f0f292a3-1e1b-49b2-a900-14563f2a9e15") : secret "dns-default-metrics-tls" not found Jul 16 12:39:42.856790 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:42.856759 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j8t4c"] Jul 16 12:39:42.949264 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:42.949178 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e2c9d30_2f89_4a05_984d_08424b633851.slice/crio-95cca0b4f2694c069c9f03622d37e875753d4643e42986f46409ee70f1f08fb3 WatchSource:0}: Error finding container 95cca0b4f2694c069c9f03622d37e875753d4643e42986f46409ee70f1f08fb3: Status 404 returned error can't find the container with id 95cca0b4f2694c069c9f03622d37e875753d4643e42986f46409ee70f1f08fb3 Jul 16 12:39:43.633914 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:43.633830 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:39:43.634379 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:43.633830 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:39:43.634379 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:43.633842 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:39:43.637072 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:43.637032 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 12:39:43.637272 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:43.637033 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 12:39:43.638165 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:43.638145 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8df2z\"" Jul 16 12:39:43.638252 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:43.638240 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 12:39:43.638313 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:43.638244 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 12:39:43.638363 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:43.638338 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ff9kh\"" Jul 16 12:39:43.739938 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:43.739901 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:39:43.740203 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:43.740072 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 12:39:43.740203 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:43.740134 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls podName:f0f292a3-1e1b-49b2-a900-14563f2a9e15 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:45.740119106 +0000 UTC m=+38.974026934 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls") pod "dns-default-67vll" (UID: "f0f292a3-1e1b-49b2-a900-14563f2a9e15") : secret "dns-default-metrics-tls" not found Jul 16 12:39:43.793204 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:43.793166 2567 generic.go:356] "Generic (PLEG): container finished" podID="5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9" containerID="e6b1e31c879fc814691305bc03f7ffde26be4a2fb0e42df29b08ae646a5198b1" exitCode=0 Jul 16 12:39:43.793359 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:43.793252 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4lc5" event={"ID":"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9","Type":"ContainerDied","Data":"e6b1e31c879fc814691305bc03f7ffde26be4a2fb0e42df29b08ae646a5198b1"} Jul 16 12:39:43.794424 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:43.794393 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j8t4c" event={"ID":"3e2c9d30-2f89-4a05-984d-08424b633851","Type":"ContainerStarted","Data":"95cca0b4f2694c069c9f03622d37e875753d4643e42986f46409ee70f1f08fb3"} Jul 16 12:39:44.801417 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:44.801378 2567 generic.go:356] "Generic (PLEG): container finished" podID="5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9" containerID="0a3948b1572d486b355e69b7a2ed0fc71a241768d9796eb1df3e7ffc7b5e2b62" exitCode=0 Jul 16 12:39:44.801838 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:44.801462 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4lc5" event={"ID":"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9","Type":"ContainerDied","Data":"0a3948b1572d486b355e69b7a2ed0fc71a241768d9796eb1df3e7ffc7b5e2b62"} Jul 16 12:39:45.756432 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:45.756250 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:39:45.756432 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:45.756409 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 12:39:45.756658 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:45.756477 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls podName:f0f292a3-1e1b-49b2-a900-14563f2a9e15 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:49.756461629 +0000 UTC m=+42.990369461 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls") pod "dns-default-67vll" (UID: "f0f292a3-1e1b-49b2-a900-14563f2a9e15") : secret "dns-default-metrics-tls" not found Jul 16 12:39:45.807336 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:45.807305 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4lc5" event={"ID":"5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9","Type":"ContainerStarted","Data":"ee6d0da9c304ae977d1890003a2c64671c83ff1bf8a10cecafe8ac4808dd6c7d"} Jul 16 12:39:45.808627 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:45.808606 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j8t4c" event={"ID":"3e2c9d30-2f89-4a05-984d-08424b633851","Type":"ContainerStarted","Data":"73457a015ccc4dc2c444b4d2183c090fd3bd1b8d1b7ad54cc716d2097e58be20"} Jul 16 12:39:45.843317 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:45.843273 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-v4lc5" podStartSLOduration=6.86046317 podStartE2EDuration="37.84325721s" totalImagesPullingTime="30.98279404s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:39:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:39:45.841472778 +0000 UTC m=+39.075380628" watchObservedRunningTime="2026-07-16 12:39:45.84325721 +0000 UTC m=+39.077165065" Jul 16 12:39:45.861106 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:45.861066 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-j8t4c" podStartSLOduration=1.750497337 podStartE2EDuration="3.861038163s" totalImagesPullingTime="2.110540826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:39:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:39:45.860209565 +0000 UTC m=+39.094117413" watchObservedRunningTime="2026-07-16 12:39:45.861038163 +0000 UTC m=+39.094946050" Jul 16 12:39:49.782636 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:49.782592 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:39:49.783407 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:49.782765 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 12:39:49.783407 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:49.782844 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls podName:f0f292a3-1e1b-49b2-a900-14563f2a9e15 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:57.782822173 +0000 UTC m=+51.016730002 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls") pod "dns-default-67vll" (UID: "f0f292a3-1e1b-49b2-a900-14563f2a9e15") : secret "dns-default-metrics-tls" not found Jul 16 12:39:51.794610 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.794579 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk"] Jul 16 12:39:51.823880 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.823851 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5d87ff988b-lrrxq"] Jul 16 12:39:51.824027 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.823967 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:39:51.827934 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.827908 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 16 12:39:51.827934 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.827915 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 12:39:51.828133 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.827915 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 12:39:51.828198 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.828182 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-88nvz\"" Jul 16 12:39:51.828385 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.828369 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 16 12:39:51.847868 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.847845 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk"] Jul 16 12:39:51.847965 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.847876 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5d87ff988b-lrrxq"] Jul 16 12:39:51.847965 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.847945 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:51.850822 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.850801 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 16 12:39:51.850947 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.850872 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vdxsj\"" Jul 16 12:39:51.850947 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.850940 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 12:39:51.851071 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.850976 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 16 12:39:51.851071 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.850926 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 12:39:51.851261 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.851244 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 16 12:39:51.851344 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.851329 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 16 12:39:51.895360 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.895339 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-stats-auth\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:51.895459 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.895365 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:51.895508 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.895455 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqlj6\" (UniqueName: \"kubernetes.io/projected/1c8777d3-5209-48ac-843c-35d07c0d30da-kube-api-access-kqlj6\") pod \"cluster-monitoring-operator-5f7f494dd4-z7mdk\" (UID: \"1c8777d3-5209-48ac-843c-35d07c0d30da\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:39:51.895508 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.895494 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1c8777d3-5209-48ac-843c-35d07c0d30da-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-z7mdk\" (UID: \"1c8777d3-5209-48ac-843c-35d07c0d30da\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:39:51.895571 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.895517 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z7mdk\" (UID: \"1c8777d3-5209-48ac-843c-35d07c0d30da\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:39:51.895571 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.895549 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-default-certificate\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:51.895571 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.895564 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:51.895663 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.895579 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2qpx\" (UniqueName: \"kubernetes.io/projected/6f912bf2-e08f-42ec-b4f2-92edf90961ba-kube-api-access-k2qpx\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:51.898232 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.898210 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf"] Jul 16 12:39:51.920516 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.920492 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf"] Jul 16 12:39:51.920622 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.920597 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" Jul 16 12:39:51.923449 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.923429 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 16 12:39:51.923557 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.923475 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-nzlhb\"" Jul 16 12:39:51.923557 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.923520 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 16 12:39:51.923557 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.923535 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 16 12:39:51.996389 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.996363 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z7mdk\" (UID: \"1c8777d3-5209-48ac-843c-35d07c0d30da\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:39:51.996505 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.996403 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-default-certificate\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:51.996505 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.996424 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:51.996505 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.996449 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2qpx\" (UniqueName: \"kubernetes.io/projected/6f912bf2-e08f-42ec-b4f2-92edf90961ba-kube-api-access-k2qpx\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:51.996646 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.996511 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdjqm\" (UniqueName: \"kubernetes.io/projected/0d16cdb2-cb20-480e-8dc5-032260738905-kube-api-access-mdjqm\") pod \"cluster-samples-operator-65db858894-dgsqf\" (UID: \"0d16cdb2-cb20-480e-8dc5-032260738905\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" Jul 16 12:39:51.996646 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:51.996518 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 12:39:51.996646 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.996557 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-stats-auth\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:51.996646 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.996579 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:51.996646 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:51.996624 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls podName:1c8777d3-5209-48ac-843c-35d07c0d30da nodeName:}" failed. No retries permitted until 2026-07-16 12:39:52.496600058 +0000 UTC m=+45.730507904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-z7mdk" (UID: "1c8777d3-5209-48ac-843c-35d07c0d30da") : secret "cluster-monitoring-operator-tls" not found Jul 16 12:39:51.996839 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:51.996650 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 12:39:51.996839 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:51.996676 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle podName:6f912bf2-e08f-42ec-b4f2-92edf90961ba nodeName:}" failed. No retries permitted until 2026-07-16 12:39:52.49666536 +0000 UTC m=+45.730573206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle") pod "router-default-5d87ff988b-lrrxq" (UID: "6f912bf2-e08f-42ec-b4f2-92edf90961ba") : configmap references non-existent config key: service-ca.crt Jul 16 12:39:51.996839 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:51.996707 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs podName:6f912bf2-e08f-42ec-b4f2-92edf90961ba nodeName:}" failed. No retries permitted until 2026-07-16 12:39:52.496690208 +0000 UTC m=+45.730598037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs") pod "router-default-5d87ff988b-lrrxq" (UID: "6f912bf2-e08f-42ec-b4f2-92edf90961ba") : secret "router-metrics-certs-default" not found Jul 16 12:39:51.996839 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.996759 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dgsqf\" (UID: \"0d16cdb2-cb20-480e-8dc5-032260738905\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" Jul 16 12:39:51.996839 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.996795 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqlj6\" (UniqueName: \"kubernetes.io/projected/1c8777d3-5209-48ac-843c-35d07c0d30da-kube-api-access-kqlj6\") pod \"cluster-monitoring-operator-5f7f494dd4-z7mdk\" (UID: \"1c8777d3-5209-48ac-843c-35d07c0d30da\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:39:51.997012 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.996841 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1c8777d3-5209-48ac-843c-35d07c0d30da-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-z7mdk\" (UID: \"1c8777d3-5209-48ac-843c-35d07c0d30da\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:39:51.997658 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.997630 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1c8777d3-5209-48ac-843c-35d07c0d30da-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-z7mdk\" (UID: \"1c8777d3-5209-48ac-843c-35d07c0d30da\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:39:51.998932 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:51.998911 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-87f8h"] Jul 16 12:39:52.000619 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.000596 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-default-certificate\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:52.000666 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.000636 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-stats-auth\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:52.013998 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.013970 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqlj6\" (UniqueName: \"kubernetes.io/projected/1c8777d3-5209-48ac-843c-35d07c0d30da-kube-api-access-kqlj6\") pod \"cluster-monitoring-operator-5f7f494dd4-z7mdk\" (UID: \"1c8777d3-5209-48ac-843c-35d07c0d30da\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:39:52.013998 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.013982 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2qpx\" (UniqueName: \"kubernetes.io/projected/6f912bf2-e08f-42ec-b4f2-92edf90961ba-kube-api-access-k2qpx\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:52.020508 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.020489 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-87f8h"] Jul 16 12:39:52.020608 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.020596 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:39:52.023330 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.023311 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 16 12:39:52.023424 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.023344 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 16 12:39:52.023424 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.023320 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 16 12:39:52.023543 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.023343 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-t6rtk\"" Jul 16 12:39:52.023543 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.023318 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 16 12:39:52.029316 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.029299 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 16 12:39:52.098214 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.098135 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/15e36e16-a26a-4369-830c-c774fde70bb8-config\") pod \"console-operator-7866497494-87f8h\" (UID: \"15e36e16-a26a-4369-830c-c774fde70bb8\") " pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:39:52.098214 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.098178 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdjqm\" (UniqueName: \"kubernetes.io/projected/0d16cdb2-cb20-480e-8dc5-032260738905-kube-api-access-mdjqm\") pod \"cluster-samples-operator-65db858894-dgsqf\" (UID: \"0d16cdb2-cb20-480e-8dc5-032260738905\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" Jul 16 12:39:52.098214 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.098196 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/15e36e16-a26a-4369-830c-c774fde70bb8-serving-cert\") pod \"console-operator-7866497494-87f8h\" (UID: \"15e36e16-a26a-4369-830c-c774fde70bb8\") " pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:39:52.098410 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.098215 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15e36e16-a26a-4369-830c-c774fde70bb8-trusted-ca\") pod \"console-operator-7866497494-87f8h\" (UID: \"15e36e16-a26a-4369-830c-c774fde70bb8\") " pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:39:52.098410 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.098280 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mdhd\" (UniqueName: \"kubernetes.io/projected/15e36e16-a26a-4369-830c-c774fde70bb8-kube-api-access-9mdhd\") pod \"console-operator-7866497494-87f8h\" (UID: \"15e36e16-a26a-4369-830c-c774fde70bb8\") " pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:39:52.098410 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.098347 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dgsqf\" (UID: \"0d16cdb2-cb20-480e-8dc5-032260738905\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" Jul 16 12:39:52.098510 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:52.098418 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 12:39:52.098510 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:52.098488 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls podName:0d16cdb2-cb20-480e-8dc5-032260738905 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:52.598469845 +0000 UTC m=+45.832377688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls") pod "cluster-samples-operator-65db858894-dgsqf" (UID: "0d16cdb2-cb20-480e-8dc5-032260738905") : secret "samples-operator-tls" not found Jul 16 12:39:52.106371 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.106345 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdjqm\" (UniqueName: \"kubernetes.io/projected/0d16cdb2-cb20-480e-8dc5-032260738905-kube-api-access-mdjqm\") pod \"cluster-samples-operator-65db858894-dgsqf\" (UID: \"0d16cdb2-cb20-480e-8dc5-032260738905\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" Jul 16 12:39:52.199357 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.199324 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/15e36e16-a26a-4369-830c-c774fde70bb8-config\") pod \"console-operator-7866497494-87f8h\" (UID: \"15e36e16-a26a-4369-830c-c774fde70bb8\") " pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:39:52.199488 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.199364 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/15e36e16-a26a-4369-830c-c774fde70bb8-serving-cert\") pod \"console-operator-7866497494-87f8h\" (UID: \"15e36e16-a26a-4369-830c-c774fde70bb8\") " pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:39:52.199488 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.199389 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15e36e16-a26a-4369-830c-c774fde70bb8-trusted-ca\") pod \"console-operator-7866497494-87f8h\" (UID: \"15e36e16-a26a-4369-830c-c774fde70bb8\") " pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:39:52.199488 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.199423 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdhd\" (UniqueName: \"kubernetes.io/projected/15e36e16-a26a-4369-830c-c774fde70bb8-kube-api-access-9mdhd\") pod \"console-operator-7866497494-87f8h\" (UID: \"15e36e16-a26a-4369-830c-c774fde70bb8\") " pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:39:52.199920 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.199900 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/15e36e16-a26a-4369-830c-c774fde70bb8-config\") pod \"console-operator-7866497494-87f8h\" (UID: \"15e36e16-a26a-4369-830c-c774fde70bb8\") " pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:39:52.200111 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.200095 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15e36e16-a26a-4369-830c-c774fde70bb8-trusted-ca\") pod \"console-operator-7866497494-87f8h\" (UID: \"15e36e16-a26a-4369-830c-c774fde70bb8\") " pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:39:52.201535 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.201516 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/15e36e16-a26a-4369-830c-c774fde70bb8-serving-cert\") pod \"console-operator-7866497494-87f8h\" (UID: \"15e36e16-a26a-4369-830c-c774fde70bb8\") " pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:39:52.206704 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.206688 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mdhd\" (UniqueName: \"kubernetes.io/projected/15e36e16-a26a-4369-830c-c774fde70bb8-kube-api-access-9mdhd\") pod \"console-operator-7866497494-87f8h\" (UID: \"15e36e16-a26a-4369-830c-c774fde70bb8\") " pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:39:52.330159 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.330125 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:39:52.498731 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.498701 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-87f8h"] Jul 16 12:39:52.501489 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.501459 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z7mdk\" (UID: \"1c8777d3-5209-48ac-843c-35d07c0d30da\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:39:52.501620 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:52.501592 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 12:39:52.501668 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.501633 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:52.501668 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:39:52.501651 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15e36e16_a26a_4369_830c_c774fde70bb8.slice/crio-cbe238d490731eb8ec90f84edf059ff10d2217bef7bc6b9c43f8f013e1e131c7 WatchSource:0}: Error finding container cbe238d490731eb8ec90f84edf059ff10d2217bef7bc6b9c43f8f013e1e131c7: Status 404 returned error can't find the container with id cbe238d490731eb8ec90f84edf059ff10d2217bef7bc6b9c43f8f013e1e131c7 Jul 16 12:39:52.501668 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:52.501660 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls podName:1c8777d3-5209-48ac-843c-35d07c0d30da nodeName:}" failed. No retries permitted until 2026-07-16 12:39:53.50164098 +0000 UTC m=+46.735548829 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-z7mdk" (UID: "1c8777d3-5209-48ac-843c-35d07c0d30da") : secret "cluster-monitoring-operator-tls" not found Jul 16 12:39:52.501784 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.501720 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:52.501784 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:52.501736 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle podName:6f912bf2-e08f-42ec-b4f2-92edf90961ba nodeName:}" failed. No retries permitted until 2026-07-16 12:39:53.501720466 +0000 UTC m=+46.735628318 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle") pod "router-default-5d87ff988b-lrrxq" (UID: "6f912bf2-e08f-42ec-b4f2-92edf90961ba") : configmap references non-existent config key: service-ca.crt Jul 16 12:39:52.501858 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:52.501798 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 12:39:52.501858 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:52.501844 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs podName:6f912bf2-e08f-42ec-b4f2-92edf90961ba nodeName:}" failed. No retries permitted until 2026-07-16 12:39:53.501830377 +0000 UTC m=+46.735738219 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs") pod "router-default-5d87ff988b-lrrxq" (UID: "6f912bf2-e08f-42ec-b4f2-92edf90961ba") : secret "router-metrics-certs-default" not found Jul 16 12:39:52.603009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.602977 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dgsqf\" (UID: \"0d16cdb2-cb20-480e-8dc5-032260738905\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" Jul 16 12:39:52.603176 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:52.603138 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 12:39:52.603215 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:52.603194 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls podName:0d16cdb2-cb20-480e-8dc5-032260738905 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:53.603180006 +0000 UTC m=+46.837087835 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls") pod "cluster-samples-operator-65db858894-dgsqf" (UID: "0d16cdb2-cb20-480e-8dc5-032260738905") : secret "samples-operator-tls" not found Jul 16 12:39:52.826160 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:52.826130 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-87f8h" event={"ID":"15e36e16-a26a-4369-830c-c774fde70bb8","Type":"ContainerStarted","Data":"cbe238d490731eb8ec90f84edf059ff10d2217bef7bc6b9c43f8f013e1e131c7"} Jul 16 12:39:53.511037 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:53.511002 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:53.511351 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:53.511146 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z7mdk\" (UID: \"1c8777d3-5209-48ac-843c-35d07c0d30da\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:39:53.511351 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:53.511179 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:53.511351 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:53.511176 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 12:39:53.511351 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:53.511262 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs podName:6f912bf2-e08f-42ec-b4f2-92edf90961ba nodeName:}" failed. No retries permitted until 2026-07-16 12:39:55.511239973 +0000 UTC m=+48.745147803 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs") pod "router-default-5d87ff988b-lrrxq" (UID: "6f912bf2-e08f-42ec-b4f2-92edf90961ba") : secret "router-metrics-certs-default" not found Jul 16 12:39:53.511351 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:53.511296 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle podName:6f912bf2-e08f-42ec-b4f2-92edf90961ba nodeName:}" failed. No retries permitted until 2026-07-16 12:39:55.511282623 +0000 UTC m=+48.745190451 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle") pod "router-default-5d87ff988b-lrrxq" (UID: "6f912bf2-e08f-42ec-b4f2-92edf90961ba") : configmap references non-existent config key: service-ca.crt Jul 16 12:39:53.511351 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:53.511297 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 12:39:53.511719 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:53.511359 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls podName:1c8777d3-5209-48ac-843c-35d07c0d30da nodeName:}" failed. No retries permitted until 2026-07-16 12:39:55.511344908 +0000 UTC m=+48.745252737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-z7mdk" (UID: "1c8777d3-5209-48ac-843c-35d07c0d30da") : secret "cluster-monitoring-operator-tls" not found Jul 16 12:39:53.612448 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:53.612411 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dgsqf\" (UID: \"0d16cdb2-cb20-480e-8dc5-032260738905\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" Jul 16 12:39:53.612635 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:53.612537 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 12:39:53.612635 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:53.612588 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls podName:0d16cdb2-cb20-480e-8dc5-032260738905 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:55.61257486 +0000 UTC m=+48.846482690 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls") pod "cluster-samples-operator-65db858894-dgsqf" (UID: "0d16cdb2-cb20-480e-8dc5-032260738905") : secret "samples-operator-tls" not found Jul 16 12:39:55.528532 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:55.528489 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z7mdk\" (UID: \"1c8777d3-5209-48ac-843c-35d07c0d30da\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:39:55.528899 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:55.528552 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:55.528899 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:55.528593 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:55.528899 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:55.528642 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 12:39:55.528899 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:55.528707 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls podName:1c8777d3-5209-48ac-843c-35d07c0d30da nodeName:}" failed. No retries permitted until 2026-07-16 12:39:59.528687896 +0000 UTC m=+52.762595745 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-z7mdk" (UID: "1c8777d3-5209-48ac-843c-35d07c0d30da") : secret "cluster-monitoring-operator-tls" not found Jul 16 12:39:55.528899 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:55.528718 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 12:39:55.528899 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:55.528734 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle podName:6f912bf2-e08f-42ec-b4f2-92edf90961ba nodeName:}" failed. No retries permitted until 2026-07-16 12:39:59.528717777 +0000 UTC m=+52.762625606 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle") pod "router-default-5d87ff988b-lrrxq" (UID: "6f912bf2-e08f-42ec-b4f2-92edf90961ba") : configmap references non-existent config key: service-ca.crt Jul 16 12:39:55.528899 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:55.528759 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs podName:6f912bf2-e08f-42ec-b4f2-92edf90961ba nodeName:}" failed. No retries permitted until 2026-07-16 12:39:59.528752716 +0000 UTC m=+52.762660545 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs") pod "router-default-5d87ff988b-lrrxq" (UID: "6f912bf2-e08f-42ec-b4f2-92edf90961ba") : secret "router-metrics-certs-default" not found Jul 16 12:39:55.629668 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:55.629634 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dgsqf\" (UID: \"0d16cdb2-cb20-480e-8dc5-032260738905\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" Jul 16 12:39:55.629824 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:55.629791 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 12:39:55.629862 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:55.629854 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls podName:0d16cdb2-cb20-480e-8dc5-032260738905 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:59.629835507 +0000 UTC m=+52.863743352 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls") pod "cluster-samples-operator-65db858894-dgsqf" (UID: "0d16cdb2-cb20-480e-8dc5-032260738905") : secret "samples-operator-tls" not found Jul 16 12:39:55.834630 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:55.834559 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/0.log" Jul 16 12:39:55.834630 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:55.834600 2567 generic.go:356] "Generic (PLEG): container finished" podID="15e36e16-a26a-4369-830c-c774fde70bb8" containerID="25f93863766b84002444eeed6a1a25a0ea00634b5b6a1218ecbb5ea8712b35ad" exitCode=255 Jul 16 12:39:55.834800 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:55.834672 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-87f8h" event={"ID":"15e36e16-a26a-4369-830c-c774fde70bb8","Type":"ContainerDied","Data":"25f93863766b84002444eeed6a1a25a0ea00634b5b6a1218ecbb5ea8712b35ad"} Jul 16 12:39:55.834932 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:55.834917 2567 scope.go:117] "RemoveContainer" containerID="25f93863766b84002444eeed6a1a25a0ea00634b5b6a1218ecbb5ea8712b35ad" Jul 16 12:39:56.838680 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:56.838654 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/1.log" Jul 16 12:39:56.839087 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:56.839070 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/0.log" Jul 16 12:39:56.839164 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:56.839106 2567 generic.go:356] "Generic (PLEG): container finished" podID="15e36e16-a26a-4369-830c-c774fde70bb8" containerID="bbed9d820c81d98f7821ea0647a99f1771d9b33c5d9256f73793c5d438c30de5" exitCode=255 Jul 16 12:39:56.839218 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:56.839167 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-87f8h" event={"ID":"15e36e16-a26a-4369-830c-c774fde70bb8","Type":"ContainerDied","Data":"bbed9d820c81d98f7821ea0647a99f1771d9b33c5d9256f73793c5d438c30de5"} Jul 16 12:39:56.839268 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:56.839229 2567 scope.go:117] "RemoveContainer" containerID="25f93863766b84002444eeed6a1a25a0ea00634b5b6a1218ecbb5ea8712b35ad" Jul 16 12:39:56.839441 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:56.839420 2567 scope.go:117] "RemoveContainer" containerID="bbed9d820c81d98f7821ea0647a99f1771d9b33c5d9256f73793c5d438c30de5" Jul 16 12:39:56.839602 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:56.839581 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-87f8h_openshift-console-operator(15e36e16-a26a-4369-830c-c774fde70bb8)\"" pod="openshift-console-operator/console-operator-7866497494-87f8h" podUID="15e36e16-a26a-4369-830c-c774fde70bb8" Jul 16 12:39:57.086320 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.086286 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-88nwz"] Jul 16 12:39:57.097526 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.097447 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:57.100918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.100896 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 12:39:57.102472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.102457 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 12:39:57.102670 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.102653 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 12:39:57.103742 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.103566 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 12:39:57.103742 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.103656 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9wswp\"" Jul 16 12:39:57.128305 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.128281 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-88nwz"] Jul 16 12:39:57.141807 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.141784 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:57.141889 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.141815 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q92dg\" (UniqueName: \"kubernetes.io/projected/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-api-access-q92dg\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:57.141889 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.141846 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bda0009f-40f8-4fa0-9c96-ef016877e844-crio-socket\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:57.141958 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.141883 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bda0009f-40f8-4fa0-9c96-ef016877e844-data-volume\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:57.141958 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.141909 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:57.243110 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.243081 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:57.243260 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.243121 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q92dg\" (UniqueName: \"kubernetes.io/projected/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-api-access-q92dg\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:57.243260 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.243153 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bda0009f-40f8-4fa0-9c96-ef016877e844-crio-socket\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:57.243260 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.243168 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bda0009f-40f8-4fa0-9c96-ef016877e844-data-volume\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:57.243260 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.243187 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:57.243260 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:57.243212 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 12:39:57.243458 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:57.243275 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm podName:bda0009f-40f8-4fa0-9c96-ef016877e844 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:57.743258457 +0000 UTC m=+50.977166287 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm") pod "insights-runtime-extractor-88nwz" (UID: "bda0009f-40f8-4fa0-9c96-ef016877e844") : configmap "kube-rbac-proxy" not found Jul 16 12:39:57.243458 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:57.243326 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 12:39:57.243458 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.243364 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bda0009f-40f8-4fa0-9c96-ef016877e844-crio-socket\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:57.243458 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:57.243383 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls podName:bda0009f-40f8-4fa0-9c96-ef016877e844 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:57.743366899 +0000 UTC m=+50.977274730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88nwz" (UID: "bda0009f-40f8-4fa0-9c96-ef016877e844") : secret "insights-runtime-extractor-tls" not found Jul 16 12:39:57.243600 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.243525 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bda0009f-40f8-4fa0-9c96-ef016877e844-data-volume\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:57.251433 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.251405 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q92dg\" (UniqueName: \"kubernetes.io/projected/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-api-access-q92dg\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:57.746268 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.746237 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:57.746439 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.746353 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:57.746439 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:57.746377 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 12:39:57.746439 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:57.746424 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 12:39:57.746439 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:57.746436 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls podName:bda0009f-40f8-4fa0-9c96-ef016877e844 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:58.746421455 +0000 UTC m=+51.980329286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88nwz" (UID: "bda0009f-40f8-4fa0-9c96-ef016877e844") : secret "insights-runtime-extractor-tls" not found Jul 16 12:39:57.746592 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:57.746461 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm podName:bda0009f-40f8-4fa0-9c96-ef016877e844 nodeName:}" failed. No retries permitted until 2026-07-16 12:39:58.746450482 +0000 UTC m=+51.980358311 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm") pod "insights-runtime-extractor-88nwz" (UID: "bda0009f-40f8-4fa0-9c96-ef016877e844") : configmap "kube-rbac-proxy" not found Jul 16 12:39:57.843429 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.843402 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/1.log" Jul 16 12:39:57.843797 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.843700 2567 scope.go:117] "RemoveContainer" containerID="bbed9d820c81d98f7821ea0647a99f1771d9b33c5d9256f73793c5d438c30de5" Jul 16 12:39:57.843849 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:57.843831 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-87f8h_openshift-console-operator(15e36e16-a26a-4369-830c-c774fde70bb8)\"" pod="openshift-console-operator/console-operator-7866497494-87f8h" podUID="15e36e16-a26a-4369-830c-c774fde70bb8" Jul 16 12:39:57.846771 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:57.846754 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:39:57.846873 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:57.846862 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 12:39:57.846923 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:57.846915 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls podName:f0f292a3-1e1b-49b2-a900-14563f2a9e15 nodeName:}" failed. No retries permitted until 2026-07-16 12:40:13.846902119 +0000 UTC m=+67.080809947 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls") pod "dns-default-67vll" (UID: "f0f292a3-1e1b-49b2-a900-14563f2a9e15") : secret "dns-default-metrics-tls" not found Jul 16 12:39:58.671251 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:58.671221 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-krt5w_a88360e6-cc58-4620-95c5-ce491044458c/dns-node-resolver/0.log" Jul 16 12:39:58.754842 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:58.754797 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:58.755016 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:58.754869 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:39:58.755016 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:58.754914 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 12:39:58.755016 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:58.754979 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm podName:bda0009f-40f8-4fa0-9c96-ef016877e844 nodeName:}" failed. No retries permitted until 2026-07-16 12:40:00.754963279 +0000 UTC m=+53.988871108 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm") pod "insights-runtime-extractor-88nwz" (UID: "bda0009f-40f8-4fa0-9c96-ef016877e844") : configmap "kube-rbac-proxy" not found Jul 16 12:39:58.755016 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:58.755011 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 12:39:58.755211 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:58.755083 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls podName:bda0009f-40f8-4fa0-9c96-ef016877e844 nodeName:}" failed. No retries permitted until 2026-07-16 12:40:00.75506688 +0000 UTC m=+53.988974714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88nwz" (UID: "bda0009f-40f8-4fa0-9c96-ef016877e844") : secret "insights-runtime-extractor-tls" not found Jul 16 12:39:59.270323 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:59.270298 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-42h29_c9ad3582-7d79-470d-a2e0-cfcb46eef262/node-ca/0.log" Jul 16 12:39:59.561288 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:59.561199 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z7mdk\" (UID: \"1c8777d3-5209-48ac-843c-35d07c0d30da\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:39:59.561288 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:59.561255 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:59.561288 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:59.561285 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:39:59.561518 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:59.561362 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 12:39:59.561518 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:59.561430 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle podName:6f912bf2-e08f-42ec-b4f2-92edf90961ba nodeName:}" failed. No retries permitted until 2026-07-16 12:40:07.561409939 +0000 UTC m=+60.795317771 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle") pod "router-default-5d87ff988b-lrrxq" (UID: "6f912bf2-e08f-42ec-b4f2-92edf90961ba") : configmap references non-existent config key: service-ca.crt Jul 16 12:39:59.561518 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:59.561432 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 12:39:59.561518 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:59.561458 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls podName:1c8777d3-5209-48ac-843c-35d07c0d30da nodeName:}" failed. No retries permitted until 2026-07-16 12:40:07.561449362 +0000 UTC m=+60.795357198 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-z7mdk" (UID: "1c8777d3-5209-48ac-843c-35d07c0d30da") : secret "cluster-monitoring-operator-tls" not found Jul 16 12:39:59.561518 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:59.561504 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs podName:6f912bf2-e08f-42ec-b4f2-92edf90961ba nodeName:}" failed. No retries permitted until 2026-07-16 12:40:07.561493296 +0000 UTC m=+60.795401125 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs") pod "router-default-5d87ff988b-lrrxq" (UID: "6f912bf2-e08f-42ec-b4f2-92edf90961ba") : secret "router-metrics-certs-default" not found Jul 16 12:39:59.662393 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:39:59.662362 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dgsqf\" (UID: \"0d16cdb2-cb20-480e-8dc5-032260738905\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" Jul 16 12:39:59.662544 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:59.662504 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 12:39:59.662583 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:39:59.662571 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls podName:0d16cdb2-cb20-480e-8dc5-032260738905 nodeName:}" failed. No retries permitted until 2026-07-16 12:40:07.662554795 +0000 UTC m=+60.896462648 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls") pod "cluster-samples-operator-65db858894-dgsqf" (UID: "0d16cdb2-cb20-480e-8dc5-032260738905") : secret "samples-operator-tls" not found Jul 16 12:40:00.072169 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:00.072139 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j8t4c_3e2c9d30-2f89-4a05-984d-08424b633851/serve-healthcheck-canary/0.log" Jul 16 12:40:00.770751 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:00.770712 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:40:00.771354 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:00.770777 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:40:00.771354 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:00.770895 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 12:40:00.771354 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:00.770920 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 12:40:00.771354 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:00.770969 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm podName:bda0009f-40f8-4fa0-9c96-ef016877e844 nodeName:}" failed. No retries permitted until 2026-07-16 12:40:04.770949534 +0000 UTC m=+58.004857366 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm") pod "insights-runtime-extractor-88nwz" (UID: "bda0009f-40f8-4fa0-9c96-ef016877e844") : configmap "kube-rbac-proxy" not found Jul 16 12:40:00.771354 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:00.770987 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls podName:bda0009f-40f8-4fa0-9c96-ef016877e844 nodeName:}" failed. No retries permitted until 2026-07-16 12:40:04.770978308 +0000 UTC m=+58.004886142 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88nwz" (UID: "bda0009f-40f8-4fa0-9c96-ef016877e844") : secret "insights-runtime-extractor-tls" not found Jul 16 12:40:02.330850 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:02.330809 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:40:02.331272 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:02.330898 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:40:02.331272 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:02.331259 2567 scope.go:117] "RemoveContainer" containerID="bbed9d820c81d98f7821ea0647a99f1771d9b33c5d9256f73793c5d438c30de5" Jul 16 12:40:02.331415 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:02.331395 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-87f8h_openshift-console-operator(15e36e16-a26a-4369-830c-c774fde70bb8)\"" pod="openshift-console-operator/console-operator-7866497494-87f8h" podUID="15e36e16-a26a-4369-830c-c774fde70bb8" Jul 16 12:40:04.804060 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:04.804000 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:40:04.804482 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:04.804100 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:40:04.804482 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:04.804126 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 12:40:04.804482 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:04.804192 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 12:40:04.804482 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:04.804206 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm podName:bda0009f-40f8-4fa0-9c96-ef016877e844 nodeName:}" failed. No retries permitted until 2026-07-16 12:40:12.80418855 +0000 UTC m=+66.038096380 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm") pod "insights-runtime-extractor-88nwz" (UID: "bda0009f-40f8-4fa0-9c96-ef016877e844") : configmap "kube-rbac-proxy" not found Jul 16 12:40:04.804482 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:04.804231 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls podName:bda0009f-40f8-4fa0-9c96-ef016877e844 nodeName:}" failed. No retries permitted until 2026-07-16 12:40:12.804220202 +0000 UTC m=+66.038128032 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88nwz" (UID: "bda0009f-40f8-4fa0-9c96-ef016877e844") : secret "insights-runtime-extractor-tls" not found Jul 16 12:40:07.628472 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:07.628428 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z7mdk\" (UID: \"1c8777d3-5209-48ac-843c-35d07c0d30da\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:40:07.628891 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:07.628484 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:40:07.628891 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:07.628512 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:40:07.628891 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:07.628597 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 12:40:07.628891 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:07.628616 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 12:40:07.628891 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:07.628654 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle podName:6f912bf2-e08f-42ec-b4f2-92edf90961ba nodeName:}" failed. No retries permitted until 2026-07-16 12:40:23.628636728 +0000 UTC m=+76.862544561 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle") pod "router-default-5d87ff988b-lrrxq" (UID: "6f912bf2-e08f-42ec-b4f2-92edf90961ba") : configmap references non-existent config key: service-ca.crt Jul 16 12:40:07.628891 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:07.628675 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls podName:1c8777d3-5209-48ac-843c-35d07c0d30da nodeName:}" failed. No retries permitted until 2026-07-16 12:40:23.628668515 +0000 UTC m=+76.862576343 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-z7mdk" (UID: "1c8777d3-5209-48ac-843c-35d07c0d30da") : secret "cluster-monitoring-operator-tls" not found Jul 16 12:40:07.628891 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:07.628686 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs podName:6f912bf2-e08f-42ec-b4f2-92edf90961ba nodeName:}" failed. No retries permitted until 2026-07-16 12:40:23.628679765 +0000 UTC m=+76.862587594 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs") pod "router-default-5d87ff988b-lrrxq" (UID: "6f912bf2-e08f-42ec-b4f2-92edf90961ba") : secret "router-metrics-certs-default" not found Jul 16 12:40:07.729247 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:07.729213 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dgsqf\" (UID: \"0d16cdb2-cb20-480e-8dc5-032260738905\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" Jul 16 12:40:07.729423 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:07.729349 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 12:40:07.729423 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:07.729413 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls podName:0d16cdb2-cb20-480e-8dc5-032260738905 nodeName:}" failed. No retries permitted until 2026-07-16 12:40:23.729387441 +0000 UTC m=+76.963295271 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls") pod "cluster-samples-operator-65db858894-dgsqf" (UID: "0d16cdb2-cb20-480e-8dc5-032260738905") : secret "samples-operator-tls" not found Jul 16 12:40:08.787359 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:08.787329 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s729s" Jul 16 12:40:12.634144 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:12.634106 2567 scope.go:117] "RemoveContainer" containerID="bbed9d820c81d98f7821ea0647a99f1771d9b33c5d9256f73793c5d438c30de5" Jul 16 12:40:12.869675 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:12.869638 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:40:12.869855 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:12.869755 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:40:12.869855 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:12.869791 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 12:40:12.869937 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:12.869863 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls podName:bda0009f-40f8-4fa0-9c96-ef016877e844 nodeName:}" failed. No retries permitted until 2026-07-16 12:40:28.869846817 +0000 UTC m=+82.103754645 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88nwz" (UID: "bda0009f-40f8-4fa0-9c96-ef016877e844") : secret "insights-runtime-extractor-tls" not found Jul 16 12:40:12.870232 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:12.870216 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bda0009f-40f8-4fa0-9c96-ef016877e844-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:40:12.877910 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:12.877891 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 12:40:12.878287 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:12.878271 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/1.log" Jul 16 12:40:12.878386 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:12.878307 2567 generic.go:356] "Generic (PLEG): container finished" podID="15e36e16-a26a-4369-830c-c774fde70bb8" containerID="1c028aefeff1676f2b7bdfff0f976758db16f145de0b9e96f4eb44e0d139f3bf" exitCode=255 Jul 16 12:40:12.878386 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:12.878356 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-87f8h" event={"ID":"15e36e16-a26a-4369-830c-c774fde70bb8","Type":"ContainerDied","Data":"1c028aefeff1676f2b7bdfff0f976758db16f145de0b9e96f4eb44e0d139f3bf"} Jul 16 12:40:12.878500 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:12.878413 2567 scope.go:117] "RemoveContainer" containerID="bbed9d820c81d98f7821ea0647a99f1771d9b33c5d9256f73793c5d438c30de5" Jul 16 12:40:12.878704 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:12.878686 2567 scope.go:117] "RemoveContainer" containerID="1c028aefeff1676f2b7bdfff0f976758db16f145de0b9e96f4eb44e0d139f3bf" Jul 16 12:40:12.878864 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:12.878846 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-87f8h_openshift-console-operator(15e36e16-a26a-4369-830c-c774fde70bb8)\"" pod="openshift-console-operator/console-operator-7866497494-87f8h" podUID="15e36e16-a26a-4369-830c-c774fde70bb8" Jul 16 12:40:12.970438 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:12.970401 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret\") pod \"global-pull-secret-syncer-78h5z\" (UID: \"f0a26b92-3040-40c7-904f-7c8112a1f5e5\") " pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:40:12.973456 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:12.973438 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 12:40:12.982977 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:12.982946 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0a26b92-3040-40c7-904f-7c8112a1f5e5-original-pull-secret\") pod \"global-pull-secret-syncer-78h5z\" (UID: \"f0a26b92-3040-40c7-904f-7c8112a1f5e5\") " pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:40:13.052615 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.052572 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78h5z" Jul 16 12:40:13.071875 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.071846 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs\") pod \"network-metrics-daemon-d6kvg\" (UID: \"8892e9d1-ce2b-47d1-af07-27d1e94dd31c\") " pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:40:13.075185 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.075163 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 12:40:13.082423 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:13.082407 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 12:40:13.082500 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:13.082476 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs podName:8892e9d1-ce2b-47d1-af07-27d1e94dd31c nodeName:}" failed. No retries permitted until 2026-07-16 12:41:17.082450757 +0000 UTC m=+130.316358588 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs") pod "network-metrics-daemon-d6kvg" (UID: "8892e9d1-ce2b-47d1-af07-27d1e94dd31c") : secret "metrics-daemon-secret" not found Jul 16 12:40:13.173311 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.173232 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkwvq\" (UniqueName: \"kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq\") pod \"network-check-target-sjhtn\" (UID: \"cf05bfdb-d809-4562-b39f-ba0dbddd0296\") " pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:40:13.176179 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.176164 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 12:40:13.180479 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.180453 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-78h5z"] Jul 16 12:40:13.183702 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:40:13.183675 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a26b92_3040_40c7_904f_7c8112a1f5e5.slice/crio-8cb8b5c6d8c782411249152f36080f5cd3de652ae59486a8c1153abc86fcc24b WatchSource:0}: Error finding container 8cb8b5c6d8c782411249152f36080f5cd3de652ae59486a8c1153abc86fcc24b: Status 404 returned error can't find the container with id 8cb8b5c6d8c782411249152f36080f5cd3de652ae59486a8c1153abc86fcc24b Jul 16 12:40:13.186016 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.185999 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 12:40:13.196111 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.196088 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkwvq\" (UniqueName: \"kubernetes.io/projected/cf05bfdb-d809-4562-b39f-ba0dbddd0296-kube-api-access-fkwvq\") pod \"network-check-target-sjhtn\" (UID: \"cf05bfdb-d809-4562-b39f-ba0dbddd0296\") " pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:40:13.349430 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.349399 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ff9kh\"" Jul 16 12:40:13.357550 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.357528 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:40:13.478755 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.478721 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sjhtn"] Jul 16 12:40:13.481646 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:40:13.481616 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf05bfdb_d809_4562_b39f_ba0dbddd0296.slice/crio-ba234769b53f70797bc6f0f67b9282d5790280cc27b70fbcb1c11e321eb284ad WatchSource:0}: Error finding container ba234769b53f70797bc6f0f67b9282d5790280cc27b70fbcb1c11e321eb284ad: Status 404 returned error can't find the container with id ba234769b53f70797bc6f0f67b9282d5790280cc27b70fbcb1c11e321eb284ad Jul 16 12:40:13.878627 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.878531 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:40:13.881503 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.881472 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0f292a3-1e1b-49b2-a900-14563f2a9e15-metrics-tls\") pod \"dns-default-67vll\" (UID: \"f0f292a3-1e1b-49b2-a900-14563f2a9e15\") " pod="openshift-dns/dns-default-67vll" Jul 16 12:40:13.882868 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.882843 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 12:40:13.884325 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.884280 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sjhtn" event={"ID":"cf05bfdb-d809-4562-b39f-ba0dbddd0296","Type":"ContainerStarted","Data":"ba234769b53f70797bc6f0f67b9282d5790280cc27b70fbcb1c11e321eb284ad"} Jul 16 12:40:13.885519 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.885492 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-78h5z" event={"ID":"f0a26b92-3040-40c7-904f-7c8112a1f5e5","Type":"ContainerStarted","Data":"8cb8b5c6d8c782411249152f36080f5cd3de652ae59486a8c1153abc86fcc24b"} Jul 16 12:40:13.933424 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.933390 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4kt9r\"" Jul 16 12:40:13.938809 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:13.938783 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-67vll" Jul 16 12:40:14.096325 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:14.096291 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-67vll"] Jul 16 12:40:14.100057 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:40:14.100012 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f292a3_1e1b_49b2_a900_14563f2a9e15.slice/crio-59b2e4127c90f18476665283ba9aea8242c8c45f4c655ac8de484406f652de98 WatchSource:0}: Error finding container 59b2e4127c90f18476665283ba9aea8242c8c45f4c655ac8de484406f652de98: Status 404 returned error can't find the container with id 59b2e4127c90f18476665283ba9aea8242c8c45f4c655ac8de484406f652de98 Jul 16 12:40:14.890461 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:14.890425 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-67vll" event={"ID":"f0f292a3-1e1b-49b2-a900-14563f2a9e15","Type":"ContainerStarted","Data":"59b2e4127c90f18476665283ba9aea8242c8c45f4c655ac8de484406f652de98"} Jul 16 12:40:18.904613 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:18.904576 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-67vll" event={"ID":"f0f292a3-1e1b-49b2-a900-14563f2a9e15","Type":"ContainerStarted","Data":"110f33bd1ad8396d435f1bf7da0d0c5f72d8eca4362cc679bf51efadee22ad70"} Jul 16 12:40:18.906313 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:18.906271 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sjhtn" event={"ID":"cf05bfdb-d809-4562-b39f-ba0dbddd0296","Type":"ContainerStarted","Data":"c2f6234cc48c83d6c5acba17393872acb78c475e0d18b9f4390e0b6e4a5527b0"} Jul 16 12:40:18.906457 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:18.906437 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:40:18.907756 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:18.907708 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-78h5z" event={"ID":"f0a26b92-3040-40c7-904f-7c8112a1f5e5","Type":"ContainerStarted","Data":"1151876bd1acc978a46b6fa958b7fb814d76cb3736d3db3b17c558af65615c2d"} Jul 16 12:40:18.927887 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:18.927845 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sjhtn" podStartSLOduration=65.734141881 podStartE2EDuration="1m10.92783268s" totalImagesPullingTime="5.193690799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:40:18.926117588 +0000 UTC m=+72.160025464" watchObservedRunningTime="2026-07-16 12:40:18.92783268 +0000 UTC m=+72.161740521" Jul 16 12:40:18.946870 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:18.946793 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-78h5z" podStartSLOduration=65.444162557 podStartE2EDuration="1m10.946771941s" totalImagesPullingTime="5.502609384s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:40:18.946417746 +0000 UTC m=+72.180325596" watchObservedRunningTime="2026-07-16 12:40:18.946771941 +0000 UTC m=+72.180679793" Jul 16 12:40:19.914790 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:19.914752 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-67vll" event={"ID":"f0f292a3-1e1b-49b2-a900-14563f2a9e15","Type":"ContainerStarted","Data":"ff471c3a07c25d71abca8524ce28381a7baaac04fa46aa0eb149fc82a3da7e93"} Jul 16 12:40:19.935567 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:19.935522 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-67vll" podStartSLOduration=33.361684185 podStartE2EDuration="37.935506893s" totalImagesPullingTime="4.573822708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:39:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:40:19.934178944 +0000 UTC m=+73.168086795" watchObservedRunningTime="2026-07-16 12:40:19.935506893 +0000 UTC m=+73.169414744" Jul 16 12:40:20.918845 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:20.918812 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-67vll" Jul 16 12:40:22.330706 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:22.330669 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:40:22.331108 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:22.330723 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:40:22.331108 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:22.330999 2567 scope.go:117] "RemoveContainer" containerID="1c028aefeff1676f2b7bdfff0f976758db16f145de0b9e96f4eb44e0d139f3bf" Jul 16 12:40:22.331186 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:22.331167 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-87f8h_openshift-console-operator(15e36e16-a26a-4369-830c-c774fde70bb8)\"" pod="openshift-console-operator/console-operator-7866497494-87f8h" podUID="15e36e16-a26a-4369-830c-c774fde70bb8" Jul 16 12:40:23.658684 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:23.658649 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z7mdk\" (UID: \"1c8777d3-5209-48ac-843c-35d07c0d30da\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:40:23.659186 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:23.658698 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:40:23.659186 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:23.658816 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:40:23.659316 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:23.659285 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f912bf2-e08f-42ec-b4f2-92edf90961ba-service-ca-bundle\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:40:23.661163 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:23.661143 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c8777d3-5209-48ac-843c-35d07c0d30da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-z7mdk\" (UID: \"1c8777d3-5209-48ac-843c-35d07c0d30da\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:40:23.661219 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:23.661199 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f912bf2-e08f-42ec-b4f2-92edf90961ba-metrics-certs\") pod \"router-default-5d87ff988b-lrrxq\" (UID: \"6f912bf2-e08f-42ec-b4f2-92edf90961ba\") " pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:40:23.760009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:23.759970 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dgsqf\" (UID: \"0d16cdb2-cb20-480e-8dc5-032260738905\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" Jul 16 12:40:23.762284 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:23.762265 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0d16cdb2-cb20-480e-8dc5-032260738905-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dgsqf\" (UID: \"0d16cdb2-cb20-480e-8dc5-032260738905\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" Jul 16 12:40:23.935503 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:23.935480 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-88nvz\"" Jul 16 12:40:23.943822 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:23.943802 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" Jul 16 12:40:23.958719 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:23.958691 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vdxsj\"" Jul 16 12:40:23.966785 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:23.966752 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:40:24.031075 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:24.031014 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-nzlhb\"" Jul 16 12:40:24.038535 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:24.038511 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" Jul 16 12:40:24.084883 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:24.084353 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk"] Jul 16 12:40:24.086984 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:40:24.086952 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c8777d3_5209_48ac_843c_35d07c0d30da.slice/crio-1d2cb56761f9eba441219b41e33a06069e131563ecf36606a270cc2781870ceb WatchSource:0}: Error finding container 1d2cb56761f9eba441219b41e33a06069e131563ecf36606a270cc2781870ceb: Status 404 returned error can't find the container with id 1d2cb56761f9eba441219b41e33a06069e131563ecf36606a270cc2781870ceb Jul 16 12:40:24.109708 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:24.109683 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5d87ff988b-lrrxq"] Jul 16 12:40:24.113786 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:40:24.113757 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f912bf2_e08f_42ec_b4f2_92edf90961ba.slice/crio-2f90216cba59805c9a78c66c02f7b9a29f63a8dbb8f303b3c13410b84d719daf WatchSource:0}: Error finding container 2f90216cba59805c9a78c66c02f7b9a29f63a8dbb8f303b3c13410b84d719daf: Status 404 returned error can't find the container with id 2f90216cba59805c9a78c66c02f7b9a29f63a8dbb8f303b3c13410b84d719daf Jul 16 12:40:24.177392 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:24.177368 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf"] Jul 16 12:40:24.932016 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:24.931980 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" event={"ID":"1c8777d3-5209-48ac-843c-35d07c0d30da","Type":"ContainerStarted","Data":"1d2cb56761f9eba441219b41e33a06069e131563ecf36606a270cc2781870ceb"} Jul 16 12:40:24.933804 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:24.933768 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5d87ff988b-lrrxq" event={"ID":"6f912bf2-e08f-42ec-b4f2-92edf90961ba","Type":"ContainerStarted","Data":"865834045cc7b12dc85ecaf1a9a710438341f540e05ff7b7f1f2105f9f3d52e2"} Jul 16 12:40:24.933945 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:24.933825 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5d87ff988b-lrrxq" event={"ID":"6f912bf2-e08f-42ec-b4f2-92edf90961ba","Type":"ContainerStarted","Data":"2f90216cba59805c9a78c66c02f7b9a29f63a8dbb8f303b3c13410b84d719daf"} Jul 16 12:40:24.935018 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:24.934995 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" event={"ID":"0d16cdb2-cb20-480e-8dc5-032260738905","Type":"ContainerStarted","Data":"c4fc359455ce3a76c5bdd1701249dff12fc1369df8c2a5dd852a40d9b2676628"} Jul 16 12:40:24.954722 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:24.954680 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5d87ff988b-lrrxq" podStartSLOduration=33.954665174 podStartE2EDuration="33.954665174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:39:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:40:24.953575243 +0000 UTC m=+78.187483095" watchObservedRunningTime="2026-07-16 12:40:24.954665174 +0000 UTC m=+78.188573028" Jul 16 12:40:24.967041 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:24.967007 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:40:24.970161 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:24.970134 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:40:25.938419 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:25.938386 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:40:25.939696 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:25.939674 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5d87ff988b-lrrxq" Jul 16 12:40:26.944642 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:26.943812 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" event={"ID":"0d16cdb2-cb20-480e-8dc5-032260738905","Type":"ContainerStarted","Data":"abbde8b8c93d836d49a5e39edb12f337d63bc878b56a415978e2d2e822dcaeb0"} Jul 16 12:40:26.944642 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:26.943879 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" event={"ID":"0d16cdb2-cb20-480e-8dc5-032260738905","Type":"ContainerStarted","Data":"7f4a09c2e2d47e906aa8fd3c736d121796ed6b1c7c660accfd6e3bd594bad008"} Jul 16 12:40:26.945616 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:26.945586 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" event={"ID":"1c8777d3-5209-48ac-843c-35d07c0d30da","Type":"ContainerStarted","Data":"a6221c542b56b3bc678ef1733e4028e926f7b45a95315dd57949ff3947f42d22"} Jul 16 12:40:26.963226 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:26.963160 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dgsqf" podStartSLOduration=33.904587577 podStartE2EDuration="35.963144161s" totalImagesPullingTime="2.058556584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:40:26.961712879 +0000 UTC m=+80.195620731" watchObservedRunningTime="2026-07-16 12:40:26.963144161 +0000 UTC m=+80.197052036" Jul 16 12:40:26.977389 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:26.977326 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-z7mdk" podStartSLOduration=33.779626844 podStartE2EDuration="35.977308438s" totalImagesPullingTime="2.197681594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:40:26.976343502 +0000 UTC m=+80.210251353" watchObservedRunningTime="2026-07-16 12:40:26.977308438 +0000 UTC m=+80.211216290" Jul 16 12:40:28.900416 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:28.900270 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:40:28.902672 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:28.902642 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bda0009f-40f8-4fa0-9c96-ef016877e844-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88nwz\" (UID: \"bda0009f-40f8-4fa0-9c96-ef016877e844\") " pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:40:28.909370 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:28.909344 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9wswp\"" Jul 16 12:40:28.917515 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:28.917495 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-88nwz" Jul 16 12:40:29.041736 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:29.041698 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-88nwz"] Jul 16 12:40:29.044431 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:40:29.044400 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda0009f_40f8_4fa0_9c96_ef016877e844.slice/crio-19d6d08f08052ea319c759000afab7212fe766b7b76842b0bee384cfc7c2f4af WatchSource:0}: Error finding container 19d6d08f08052ea319c759000afab7212fe766b7b76842b0bee384cfc7c2f4af: Status 404 returned error can't find the container with id 19d6d08f08052ea319c759000afab7212fe766b7b76842b0bee384cfc7c2f4af Jul 16 12:40:29.957629 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:29.957591 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-88nwz" event={"ID":"bda0009f-40f8-4fa0-9c96-ef016877e844","Type":"ContainerStarted","Data":"19d6d08f08052ea319c759000afab7212fe766b7b76842b0bee384cfc7c2f4af"} Jul 16 12:40:30.496276 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.496238 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-wpzqj"] Jul 16 12:40:30.498949 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.498918 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-wpzqj" Jul 16 12:40:30.503371 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.503329 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-5fhdh\"" Jul 16 12:40:30.503509 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.503437 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 16 12:40:30.503821 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.503788 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 16 12:40:30.504766 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.504719 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj"] Jul 16 12:40:30.507455 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.507041 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj" Jul 16 12:40:30.510246 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.510205 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 16 12:40:30.510476 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.510447 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 16 12:40:30.510579 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.510555 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 16 12:40:30.510684 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.510669 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 16 12:40:30.516024 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.515981 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-wpzqj"] Jul 16 12:40:30.532291 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.532258 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj"] Jul 16 12:40:30.538992 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.538967 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76d4b7d8fd-2b8r7"] Jul 16 12:40:30.544123 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.543144 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.546992 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.546967 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 12:40:30.547164 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.546980 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zz4g5\"" Jul 16 12:40:30.548013 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.547995 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 12:40:30.548431 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.548214 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 12:40:30.553465 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.553446 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 12:40:30.556935 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.556912 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76d4b7d8fd-2b8r7"] Jul 16 12:40:30.614363 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.614324 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/06136c04-642c-4fc9-806b-40424fd711bb-nginx-conf\") pod \"networking-console-plugin-6cf856758c-wpzqj\" (UID: \"06136c04-642c-4fc9-806b-40424fd711bb\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wpzqj" Jul 16 12:40:30.614551 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.614391 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5htlj\" (UniqueName: \"kubernetes.io/projected/a79f63fe-0c9d-432e-b119-b707439a2d11-kube-api-access-5htlj\") pod \"klusterlet-addon-workmgr-6c895b96b4-g6mdj\" (UID: \"a79f63fe-0c9d-432e-b119-b707439a2d11\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj" Jul 16 12:40:30.614551 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.614427 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06136c04-642c-4fc9-806b-40424fd711bb-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wpzqj\" (UID: \"06136c04-642c-4fc9-806b-40424fd711bb\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wpzqj" Jul 16 12:40:30.614551 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.614528 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a79f63fe-0c9d-432e-b119-b707439a2d11-tmp\") pod \"klusterlet-addon-workmgr-6c895b96b4-g6mdj\" (UID: \"a79f63fe-0c9d-432e-b119-b707439a2d11\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj" Jul 16 12:40:30.614680 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.614561 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a79f63fe-0c9d-432e-b119-b707439a2d11-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c895b96b4-g6mdj\" (UID: \"a79f63fe-0c9d-432e-b119-b707439a2d11\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj" Jul 16 12:40:30.715894 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.715855 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njg49\" (UniqueName: \"kubernetes.io/projected/88164a6d-4e64-4b9e-9805-400ded0814f1-kube-api-access-njg49\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.716126 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.715905 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a79f63fe-0c9d-432e-b119-b707439a2d11-tmp\") pod \"klusterlet-addon-workmgr-6c895b96b4-g6mdj\" (UID: \"a79f63fe-0c9d-432e-b119-b707439a2d11\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj" Jul 16 12:40:30.716126 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.715948 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88164a6d-4e64-4b9e-9805-400ded0814f1-image-registry-private-configuration\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.716126 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.715977 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88164a6d-4e64-4b9e-9805-400ded0814f1-registry-tls\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.716126 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.715999 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88164a6d-4e64-4b9e-9805-400ded0814f1-installation-pull-secrets\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.716126 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.716025 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a79f63fe-0c9d-432e-b119-b707439a2d11-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c895b96b4-g6mdj\" (UID: \"a79f63fe-0c9d-432e-b119-b707439a2d11\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj" Jul 16 12:40:30.716347 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.716218 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/06136c04-642c-4fc9-806b-40424fd711bb-nginx-conf\") pod \"networking-console-plugin-6cf856758c-wpzqj\" (UID: \"06136c04-642c-4fc9-806b-40424fd711bb\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wpzqj" Jul 16 12:40:30.716347 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.716265 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88164a6d-4e64-4b9e-9805-400ded0814f1-ca-trust-extracted\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.716347 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.716308 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5htlj\" (UniqueName: \"kubernetes.io/projected/a79f63fe-0c9d-432e-b119-b707439a2d11-kube-api-access-5htlj\") pod \"klusterlet-addon-workmgr-6c895b96b4-g6mdj\" (UID: \"a79f63fe-0c9d-432e-b119-b707439a2d11\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj" Jul 16 12:40:30.716479 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.716349 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a79f63fe-0c9d-432e-b119-b707439a2d11-tmp\") pod \"klusterlet-addon-workmgr-6c895b96b4-g6mdj\" (UID: \"a79f63fe-0c9d-432e-b119-b707439a2d11\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj" Jul 16 12:40:30.716479 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.716358 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06136c04-642c-4fc9-806b-40424fd711bb-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wpzqj\" (UID: \"06136c04-642c-4fc9-806b-40424fd711bb\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wpzqj" Jul 16 12:40:30.716479 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.716409 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88164a6d-4e64-4b9e-9805-400ded0814f1-trusted-ca\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.716479 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.716454 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88164a6d-4e64-4b9e-9805-400ded0814f1-registry-certificates\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.716655 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.716555 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88164a6d-4e64-4b9e-9805-400ded0814f1-bound-sa-token\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.717040 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.716960 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/06136c04-642c-4fc9-806b-40424fd711bb-nginx-conf\") pod \"networking-console-plugin-6cf856758c-wpzqj\" (UID: \"06136c04-642c-4fc9-806b-40424fd711bb\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wpzqj" Jul 16 12:40:30.719287 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.719240 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06136c04-642c-4fc9-806b-40424fd711bb-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wpzqj\" (UID: \"06136c04-642c-4fc9-806b-40424fd711bb\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wpzqj" Jul 16 12:40:30.719413 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.719285 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a79f63fe-0c9d-432e-b119-b707439a2d11-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c895b96b4-g6mdj\" (UID: \"a79f63fe-0c9d-432e-b119-b707439a2d11\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj" Jul 16 12:40:30.728269 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.728237 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5htlj\" (UniqueName: \"kubernetes.io/projected/a79f63fe-0c9d-432e-b119-b707439a2d11-kube-api-access-5htlj\") pod \"klusterlet-addon-workmgr-6c895b96b4-g6mdj\" (UID: \"a79f63fe-0c9d-432e-b119-b707439a2d11\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj" Jul 16 12:40:30.811888 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.811792 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-wpzqj" Jul 16 12:40:30.817244 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.817210 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88164a6d-4e64-4b9e-9805-400ded0814f1-registry-certificates\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.817375 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.817282 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88164a6d-4e64-4b9e-9805-400ded0814f1-bound-sa-token\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.817375 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.817317 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njg49\" (UniqueName: \"kubernetes.io/projected/88164a6d-4e64-4b9e-9805-400ded0814f1-kube-api-access-njg49\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.817375 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.817357 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88164a6d-4e64-4b9e-9805-400ded0814f1-image-registry-private-configuration\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.817540 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.817386 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88164a6d-4e64-4b9e-9805-400ded0814f1-registry-tls\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.817540 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.817410 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88164a6d-4e64-4b9e-9805-400ded0814f1-installation-pull-secrets\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.817540 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.817444 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88164a6d-4e64-4b9e-9805-400ded0814f1-ca-trust-extracted\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.817540 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.817485 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88164a6d-4e64-4b9e-9805-400ded0814f1-trusted-ca\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.818099 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.818078 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88164a6d-4e64-4b9e-9805-400ded0814f1-registry-certificates\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.818412 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.818389 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88164a6d-4e64-4b9e-9805-400ded0814f1-trusted-ca\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.818858 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.818825 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88164a6d-4e64-4b9e-9805-400ded0814f1-ca-trust-extracted\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.821241 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.821194 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88164a6d-4e64-4b9e-9805-400ded0814f1-image-registry-private-configuration\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.821362 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.821315 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88164a6d-4e64-4b9e-9805-400ded0814f1-installation-pull-secrets\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.821491 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.821325 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88164a6d-4e64-4b9e-9805-400ded0814f1-registry-tls\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.822229 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.822033 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj" Jul 16 12:40:30.826573 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.826537 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88164a6d-4e64-4b9e-9805-400ded0814f1-bound-sa-token\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.826876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.826851 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njg49\" (UniqueName: \"kubernetes.io/projected/88164a6d-4e64-4b9e-9805-400ded0814f1-kube-api-access-njg49\") pod \"image-registry-76d4b7d8fd-2b8r7\" (UID: \"88164a6d-4e64-4b9e-9805-400ded0814f1\") " pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.860080 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.857237 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:30.927245 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:30.925945 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-67vll" Jul 16 12:40:31.001981 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:31.001866 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-wpzqj"] Jul 16 12:40:31.005807 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:40:31.005760 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06136c04_642c_4fc9_806b_40424fd711bb.slice/crio-2924e579b049fba0df7a555bdeaf2b6958fa26e278f4656bfdc1309bcf984ac4 WatchSource:0}: Error finding container 2924e579b049fba0df7a555bdeaf2b6958fa26e278f4656bfdc1309bcf984ac4: Status 404 returned error can't find the container with id 2924e579b049fba0df7a555bdeaf2b6958fa26e278f4656bfdc1309bcf984ac4 Jul 16 12:40:31.037279 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:31.037175 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj"] Jul 16 12:40:31.041478 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:40:31.041443 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda79f63fe_0c9d_432e_b119_b707439a2d11.slice/crio-e5d1aa12a3519008f553c412fb758172dda2fe2d98ff4326e0de1ddb45bd5f48 WatchSource:0}: Error finding container e5d1aa12a3519008f553c412fb758172dda2fe2d98ff4326e0de1ddb45bd5f48: Status 404 returned error can't find the container with id e5d1aa12a3519008f553c412fb758172dda2fe2d98ff4326e0de1ddb45bd5f48 Jul 16 12:40:31.060394 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:31.060362 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76d4b7d8fd-2b8r7"] Jul 16 12:40:31.063173 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:40:31.063099 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88164a6d_4e64_4b9e_9805_400ded0814f1.slice/crio-596482cabe51510934850dcf80c02caa002abe0fd2753b07b2501984ab1ef3ff WatchSource:0}: Error finding container 596482cabe51510934850dcf80c02caa002abe0fd2753b07b2501984ab1ef3ff: Status 404 returned error can't find the container with id 596482cabe51510934850dcf80c02caa002abe0fd2753b07b2501984ab1ef3ff Jul 16 12:40:31.969926 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:31.969838 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj" event={"ID":"a79f63fe-0c9d-432e-b119-b707439a2d11","Type":"ContainerStarted","Data":"e5d1aa12a3519008f553c412fb758172dda2fe2d98ff4326e0de1ddb45bd5f48"} Jul 16 12:40:31.971447 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:31.971373 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-wpzqj" event={"ID":"06136c04-642c-4fc9-806b-40424fd711bb","Type":"ContainerStarted","Data":"2924e579b049fba0df7a555bdeaf2b6958fa26e278f4656bfdc1309bcf984ac4"} Jul 16 12:40:31.974062 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:31.973614 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" event={"ID":"88164a6d-4e64-4b9e-9805-400ded0814f1","Type":"ContainerStarted","Data":"230eb47c66952575431bf6b02ee6fd912ad6c39574195568d61099bffdb2eda5"} Jul 16 12:40:31.974062 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:31.973662 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" event={"ID":"88164a6d-4e64-4b9e-9805-400ded0814f1","Type":"ContainerStarted","Data":"596482cabe51510934850dcf80c02caa002abe0fd2753b07b2501984ab1ef3ff"} Jul 16 12:40:31.974062 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:31.973956 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:32.001887 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:32.001825 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" podStartSLOduration=2.001808584 podStartE2EDuration="2.001808584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:40:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:40:32.000976073 +0000 UTC m=+85.234883954" watchObservedRunningTime="2026-07-16 12:40:32.001808584 +0000 UTC m=+85.235716435" Jul 16 12:40:34.836166 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:34.836121 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-vtj5k"] Jul 16 12:40:34.842146 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:34.842122 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" Jul 16 12:40:34.847714 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:34.847436 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 16 12:40:34.847714 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:34.847453 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 12:40:34.847714 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:34.847467 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-7lpdm\"" Jul 16 12:40:34.847959 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:34.847801 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 16 12:40:34.849319 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:34.849288 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-vtj5k"] Jul 16 12:40:34.953970 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:34.953932 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4b2d8894-e54d-4464-81b6-0575052d69b8-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-vtj5k\" (UID: \"4b2d8894-e54d-4464-81b6-0575052d69b8\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" Jul 16 12:40:34.954174 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:34.954004 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb7qk\" (UniqueName: \"kubernetes.io/projected/4b2d8894-e54d-4464-81b6-0575052d69b8-kube-api-access-lb7qk\") pod \"prometheus-operator-64785b6675-vtj5k\" (UID: \"4b2d8894-e54d-4464-81b6-0575052d69b8\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" Jul 16 12:40:34.954174 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:34.954098 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b2d8894-e54d-4464-81b6-0575052d69b8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-vtj5k\" (UID: \"4b2d8894-e54d-4464-81b6-0575052d69b8\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" Jul 16 12:40:34.954174 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:34.954138 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b2d8894-e54d-4464-81b6-0575052d69b8-metrics-client-ca\") pod \"prometheus-operator-64785b6675-vtj5k\" (UID: \"4b2d8894-e54d-4464-81b6-0575052d69b8\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" Jul 16 12:40:35.055198 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:35.055161 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb7qk\" (UniqueName: \"kubernetes.io/projected/4b2d8894-e54d-4464-81b6-0575052d69b8-kube-api-access-lb7qk\") pod \"prometheus-operator-64785b6675-vtj5k\" (UID: \"4b2d8894-e54d-4464-81b6-0575052d69b8\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" Jul 16 12:40:35.055372 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:35.055246 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b2d8894-e54d-4464-81b6-0575052d69b8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-vtj5k\" (UID: \"4b2d8894-e54d-4464-81b6-0575052d69b8\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" Jul 16 12:40:35.055372 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:35.055280 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b2d8894-e54d-4464-81b6-0575052d69b8-metrics-client-ca\") pod \"prometheus-operator-64785b6675-vtj5k\" (UID: \"4b2d8894-e54d-4464-81b6-0575052d69b8\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" Jul 16 12:40:35.055372 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:35.055348 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4b2d8894-e54d-4464-81b6-0575052d69b8-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-vtj5k\" (UID: \"4b2d8894-e54d-4464-81b6-0575052d69b8\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" Jul 16 12:40:35.056127 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:35.056101 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b2d8894-e54d-4464-81b6-0575052d69b8-metrics-client-ca\") pod \"prometheus-operator-64785b6675-vtj5k\" (UID: \"4b2d8894-e54d-4464-81b6-0575052d69b8\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" Jul 16 12:40:35.058140 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:35.058116 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4b2d8894-e54d-4464-81b6-0575052d69b8-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-vtj5k\" (UID: \"4b2d8894-e54d-4464-81b6-0575052d69b8\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" Jul 16 12:40:35.058240 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:35.058116 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b2d8894-e54d-4464-81b6-0575052d69b8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-vtj5k\" (UID: \"4b2d8894-e54d-4464-81b6-0575052d69b8\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" Jul 16 12:40:35.063797 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:35.063749 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb7qk\" (UniqueName: \"kubernetes.io/projected/4b2d8894-e54d-4464-81b6-0575052d69b8-kube-api-access-lb7qk\") pod \"prometheus-operator-64785b6675-vtj5k\" (UID: \"4b2d8894-e54d-4464-81b6-0575052d69b8\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" Jul 16 12:40:35.154309 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:35.154232 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" Jul 16 12:40:37.094534 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:37.094446 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-vtj5k"] Jul 16 12:40:37.097675 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:40:37.097645 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b2d8894_e54d_4464_81b6_0575052d69b8.slice/crio-3f7907dffb88e2b4d88b9b3c9bf5062dccd80cf31a6169c66db52c7c3e78ca34 WatchSource:0}: Error finding container 3f7907dffb88e2b4d88b9b3c9bf5062dccd80cf31a6169c66db52c7c3e78ca34: Status 404 returned error can't find the container with id 3f7907dffb88e2b4d88b9b3c9bf5062dccd80cf31a6169c66db52c7c3e78ca34 Jul 16 12:40:37.635956 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:37.635563 2567 scope.go:117] "RemoveContainer" containerID="1c028aefeff1676f2b7bdfff0f976758db16f145de0b9e96f4eb44e0d139f3bf" Jul 16 12:40:38.003765 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:38.003733 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 12:40:38.003952 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:38.003847 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-87f8h" event={"ID":"15e36e16-a26a-4369-830c-c774fde70bb8","Type":"ContainerStarted","Data":"ddf6cfa37a56dba5edb15d4c139ca4d624e75992ff41cf2e01dc6a172649ee78"} Jul 16 12:40:38.004245 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:38.004189 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:40:38.007237 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:38.006713 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj" event={"ID":"a79f63fe-0c9d-432e-b119-b707439a2d11","Type":"ContainerStarted","Data":"1f621c1a1e727e1178b73775dd89db680e242d96d3927a00761073b5e76ad5ff"} Jul 16 12:40:38.007377 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:38.007363 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj" Jul 16 12:40:38.008912 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:38.008881 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj" Jul 16 12:40:38.009452 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:38.009428 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-wpzqj" event={"ID":"06136c04-642c-4fc9-806b-40424fd711bb","Type":"ContainerStarted","Data":"3e6d83885ebb52ad9e3aa430f47989139da06e21eb97c623c86cde09c7656f7d"} Jul 16 12:40:38.013130 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:38.013105 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-88nwz" event={"ID":"bda0009f-40f8-4fa0-9c96-ef016877e844","Type":"ContainerStarted","Data":"395a8b9511860d428b19c5e5157e34e9b9a57e80962aee388e6fc27c8a7a9a97"} Jul 16 12:40:38.015084 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:38.015058 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" event={"ID":"4b2d8894-e54d-4464-81b6-0575052d69b8","Type":"ContainerStarted","Data":"3f7907dffb88e2b4d88b9b3c9bf5062dccd80cf31a6169c66db52c7c3e78ca34"} Jul 16 12:40:38.026699 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:38.026644 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-87f8h" podStartSLOduration=44.754496609 podStartE2EDuration="47.026604288s" totalImagesPullingTime="2.272107679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:40:38.026091245 +0000 UTC m=+91.259999097" watchObservedRunningTime="2026-07-16 12:40:38.026604288 +0000 UTC m=+91.260512140" Jul 16 12:40:38.044558 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:38.044178 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c895b96b4-g6mdj" podStartSLOduration=2.116162519 podStartE2EDuration="8.044160006s" totalImagesPullingTime="5.927997487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:40:38.043765054 +0000 UTC m=+91.277672906" watchObservedRunningTime="2026-07-16 12:40:38.044160006 +0000 UTC m=+91.278067854" Jul 16 12:40:38.062713 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:38.062659 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-wpzqj" podStartSLOduration=2.121732857 podStartE2EDuration="8.062639634s" totalImagesPullingTime="5.940906777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:40:38.062509516 +0000 UTC m=+91.296417368" watchObservedRunningTime="2026-07-16 12:40:38.062639634 +0000 UTC m=+91.296547486" Jul 16 12:40:38.914547 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:38.914463 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-87f8h" Jul 16 12:40:39.021992 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:39.021950 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" event={"ID":"4b2d8894-e54d-4464-81b6-0575052d69b8","Type":"ContainerStarted","Data":"705b6f4d984bc9e4f4202cf63670dcf9e54fbfbf23d83de18d2a6e79c839d11e"} Jul 16 12:40:39.022154 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:39.022021 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" event={"ID":"4b2d8894-e54d-4464-81b6-0575052d69b8","Type":"ContainerStarted","Data":"bfba1fa0a093d4b2d6bb9c574dfae3db474ee1ca87a486fdde43bcc4b40a5cc5"} Jul 16 12:40:39.040288 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:39.040218 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-vtj5k" podStartSLOduration=3.58311105 podStartE2EDuration="5.040196514s" totalImagesPullingTime="1.457085464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:40:39.039739394 +0000 UTC m=+92.273647246" watchObservedRunningTime="2026-07-16 12:40:39.040196514 +0000 UTC m=+92.274104366" Jul 16 12:40:41.032544 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.032501 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-88nwz" event={"ID":"bda0009f-40f8-4fa0-9c96-ef016877e844","Type":"ContainerStarted","Data":"469c4a9741498bb7bb757a657ed0b54fd6a29a5a2206cddcf1d10bcd7145ced0"} Jul 16 12:40:41.303466 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.303429 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-s5w6f"] Jul 16 12:40:41.306623 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.306605 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-rnwsr"] Jul 16 12:40:41.306754 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.306736 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.309296 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.309269 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jsphr\"" Jul 16 12:40:41.309296 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.309298 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 12:40:41.309487 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.309362 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 12:40:41.309487 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.309381 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 12:40:41.309598 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.309570 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8ad22e82-e80f-41b2-b40c-a9562db3e320-node-exporter-wtmp\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.309668 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.309613 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ad22e82-e80f-41b2-b40c-a9562db3e320-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.309668 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.309652 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8ad22e82-e80f-41b2-b40c-a9562db3e320-root\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.309747 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.309710 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8ad22e82-e80f-41b2-b40c-a9562db3e320-node-exporter-tls\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.309747 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.309715 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.309825 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.309741 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8ad22e82-e80f-41b2-b40c-a9562db3e320-node-exporter-accelerators-collector-config\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.309825 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.309772 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ad22e82-e80f-41b2-b40c-a9562db3e320-sys\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.309825 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.309812 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ad22e82-e80f-41b2-b40c-a9562db3e320-metrics-client-ca\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.309976 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.309852 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brp9t\" (UniqueName: \"kubernetes.io/projected/8ad22e82-e80f-41b2-b40c-a9562db3e320-kube-api-access-brp9t\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.309976 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.309935 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8ad22e82-e80f-41b2-b40c-a9562db3e320-node-exporter-textfile\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.313097 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.312909 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 16 12:40:41.313097 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.312933 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 16 12:40:41.313097 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.312964 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 16 12:40:41.314015 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.313996 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-6kcvw\"" Jul 16 12:40:41.317722 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.317701 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-rnwsr"] Jul 16 12:40:41.411199 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411167 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.411388 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411215 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.411388 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411251 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8ad22e82-e80f-41b2-b40c-a9562db3e320-node-exporter-textfile\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.411388 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411276 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.411388 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411304 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.411388 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411369 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8ad22e82-e80f-41b2-b40c-a9562db3e320-node-exporter-wtmp\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.411592 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411410 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lg56\" (UniqueName: \"kubernetes.io/projected/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-api-access-5lg56\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.411592 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411443 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ad22e82-e80f-41b2-b40c-a9562db3e320-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.411592 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411462 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8ad22e82-e80f-41b2-b40c-a9562db3e320-root\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.411592 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411475 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8ad22e82-e80f-41b2-b40c-a9562db3e320-node-exporter-wtmp\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.411592 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411512 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8ad22e82-e80f-41b2-b40c-a9562db3e320-root\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.411592 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411542 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8ad22e82-e80f-41b2-b40c-a9562db3e320-node-exporter-tls\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.411592 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411558 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8ad22e82-e80f-41b2-b40c-a9562db3e320-node-exporter-accelerators-collector-config\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.411592 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411575 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8ad22e82-e80f-41b2-b40c-a9562db3e320-node-exporter-textfile\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.411592 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411581 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ad22e82-e80f-41b2-b40c-a9562db3e320-sys\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.411995 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411609 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ad22e82-e80f-41b2-b40c-a9562db3e320-sys\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.411995 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411615 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ad22e82-e80f-41b2-b40c-a9562db3e320-metrics-client-ca\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.411995 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411764 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brp9t\" (UniqueName: \"kubernetes.io/projected/8ad22e82-e80f-41b2-b40c-a9562db3e320-kube-api-access-brp9t\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.411995 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.411798 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.412223 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.412175 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8ad22e82-e80f-41b2-b40c-a9562db3e320-node-exporter-accelerators-collector-config\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.412273 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.412254 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ad22e82-e80f-41b2-b40c-a9562db3e320-metrics-client-ca\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.414117 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.414093 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8ad22e82-e80f-41b2-b40c-a9562db3e320-node-exporter-tls\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.414215 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.414096 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ad22e82-e80f-41b2-b40c-a9562db3e320-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.421841 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.421819 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brp9t\" (UniqueName: \"kubernetes.io/projected/8ad22e82-e80f-41b2-b40c-a9562db3e320-kube-api-access-brp9t\") pod \"node-exporter-s5w6f\" (UID: \"8ad22e82-e80f-41b2-b40c-a9562db3e320\") " pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.512136 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.512101 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.512136 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.512139 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.512314 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.512172 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.512314 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.512203 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.512314 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.512236 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.512314 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:41.512249 2567 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 16 12:40:41.512314 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.512280 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lg56\" (UniqueName: \"kubernetes.io/projected/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-api-access-5lg56\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.512506 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:41.512318 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-state-metrics-tls podName:d97e9e1d-2485-429c-9e77-6dd6fa67d02d nodeName:}" failed. No retries permitted until 2026-07-16 12:40:42.012297717 +0000 UTC m=+95.246205549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-rnwsr" (UID: "d97e9e1d-2485-429c-9e77-6dd6fa67d02d") : secret "kube-state-metrics-tls" not found Jul 16 12:40:41.512806 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.512780 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.513006 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.512987 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.513097 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.512993 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.514641 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.514620 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.523918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.523895 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lg56\" (UniqueName: \"kubernetes.io/projected/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-api-access-5lg56\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:41.619638 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:41.619550 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s5w6f" Jul 16 12:40:41.633299 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:40:41.633264 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad22e82_e80f_41b2_b40c_a9562db3e320.slice/crio-caf80229ba538a5bed55933bbad0eaef7ebb46e68de810a772d585ea79b2b244 WatchSource:0}: Error finding container caf80229ba538a5bed55933bbad0eaef7ebb46e68de810a772d585ea79b2b244: Status 404 returned error can't find the container with id caf80229ba538a5bed55933bbad0eaef7ebb46e68de810a772d585ea79b2b244 Jul 16 12:40:42.015498 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:42.015454 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:42.015680 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:42.015616 2567 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 16 12:40:42.015734 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:40:42.015709 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-state-metrics-tls podName:d97e9e1d-2485-429c-9e77-6dd6fa67d02d nodeName:}" failed. No retries permitted until 2026-07-16 12:40:43.015684676 +0000 UTC m=+96.249592504 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-rnwsr" (UID: "d97e9e1d-2485-429c-9e77-6dd6fa67d02d") : secret "kube-state-metrics-tls" not found Jul 16 12:40:42.038685 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:42.038648 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s5w6f" event={"ID":"8ad22e82-e80f-41b2-b40c-a9562db3e320","Type":"ContainerStarted","Data":"caf80229ba538a5bed55933bbad0eaef7ebb46e68de810a772d585ea79b2b244"} Jul 16 12:40:43.023829 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:43.023791 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:43.026562 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:43.026531 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d97e9e1d-2485-429c-9e77-6dd6fa67d02d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-rnwsr\" (UID: \"d97e9e1d-2485-429c-9e77-6dd6fa67d02d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:43.124893 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:43.124849 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" Jul 16 12:40:45.046450 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:45.046423 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-rnwsr"] Jul 16 12:40:45.051066 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:40:45.051016 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd97e9e1d_2485_429c_9e77_6dd6fa67d02d.slice/crio-9815d631c8e1c78be0dc641c48004fbbb7777e0c93073e93a2281f9d85bea242 WatchSource:0}: Error finding container 9815d631c8e1c78be0dc641c48004fbbb7777e0c93073e93a2281f9d85bea242: Status 404 returned error can't find the container with id 9815d631c8e1c78be0dc641c48004fbbb7777e0c93073e93a2281f9d85bea242 Jul 16 12:40:45.054589 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:45.054095 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-88nwz" event={"ID":"bda0009f-40f8-4fa0-9c96-ef016877e844","Type":"ContainerStarted","Data":"e90dd15cec81420a9f1b3c0c64392f5ed43bab41b1f435098569a474a42bcd2f"} Jul 16 12:40:45.056418 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:45.056395 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s5w6f" event={"ID":"8ad22e82-e80f-41b2-b40c-a9562db3e320","Type":"ContainerStarted","Data":"0f613a2041e040db2c7bcab1fac590fe7ae854aa6d94412f0dad5794a24cfda8"} Jul 16 12:40:45.075191 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:45.075127 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-88nwz" podStartSLOduration=32.337616884 podStartE2EDuration="48.075107463s" totalImagesPullingTime="15.737490579s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:39:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:40:45.073723239 +0000 UTC m=+98.307631090" watchObservedRunningTime="2026-07-16 12:40:45.075107463 +0000 UTC m=+98.309015316" Jul 16 12:40:46.063020 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:46.062983 2567 generic.go:356] "Generic (PLEG): container finished" podID="8ad22e82-e80f-41b2-b40c-a9562db3e320" containerID="0f613a2041e040db2c7bcab1fac590fe7ae854aa6d94412f0dad5794a24cfda8" exitCode=0 Jul 16 12:40:46.063441 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:46.063080 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s5w6f" event={"ID":"8ad22e82-e80f-41b2-b40c-a9562db3e320","Type":"ContainerDied","Data":"0f613a2041e040db2c7bcab1fac590fe7ae854aa6d94412f0dad5794a24cfda8"} Jul 16 12:40:46.064684 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:46.064651 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" event={"ID":"d97e9e1d-2485-429c-9e77-6dd6fa67d02d","Type":"ContainerStarted","Data":"9815d631c8e1c78be0dc641c48004fbbb7777e0c93073e93a2281f9d85bea242"} Jul 16 12:40:47.071150 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.071112 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s5w6f" event={"ID":"8ad22e82-e80f-41b2-b40c-a9562db3e320","Type":"ContainerStarted","Data":"864ed429b2dae5f53581e14b513626966befff2229c098393bf3bbd5a646263f"} Jul 16 12:40:47.071571 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.071172 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s5w6f" event={"ID":"8ad22e82-e80f-41b2-b40c-a9562db3e320","Type":"ContainerStarted","Data":"0f92899c2d596bf6fe5fee993b77c9c321dffe639a9dab0ca09fdfa1260419d1"} Jul 16 12:40:47.073139 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.073112 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" event={"ID":"d97e9e1d-2485-429c-9e77-6dd6fa67d02d","Type":"ContainerStarted","Data":"909ae4e7ff22a1000a57a837c367c38b903c9ff0415c33a3a5a06c09c3c4c1ed"} Jul 16 12:40:47.073244 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.073160 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" event={"ID":"d97e9e1d-2485-429c-9e77-6dd6fa67d02d","Type":"ContainerStarted","Data":"655560db3ef6414ee1ace489fefc975575be0e00197e920689675a864a3076b1"} Jul 16 12:40:47.073244 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.073178 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" event={"ID":"d97e9e1d-2485-429c-9e77-6dd6fa67d02d","Type":"ContainerStarted","Data":"48eebcaeb44a4c2e472b81b1321b3fb9857c1bd33096f5ef6d624cb687617610"} Jul 16 12:40:47.095660 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.095613 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-s5w6f" podStartSLOduration=2.752536314 podStartE2EDuration="6.095599594s" totalImagesPullingTime="3.275193241s" totalInitContainerRuntime="67.870039ms" isStatefulPod=true podCreationTimestamp="2026-07-16 12:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:40:47.093649753 +0000 UTC m=+100.327557604" watchObservedRunningTime="2026-07-16 12:40:47.095599594 +0000 UTC m=+100.329507445" Jul 16 12:40:47.113962 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.113922 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-rnwsr" podStartSLOduration=4.84533868 podStartE2EDuration="6.113910151s" totalImagesPullingTime="1.268571471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:40:47.113004986 +0000 UTC m=+100.346912849" watchObservedRunningTime="2026-07-16 12:40:47.113910151 +0000 UTC m=+100.347818001" Jul 16 12:40:47.843749 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.843715 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f5cd5fdbf-d7db9"] Jul 16 12:40:47.847206 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.847182 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.851139 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.851104 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 12:40:47.851264 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.851248 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 12:40:47.851327 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.851266 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 12:40:47.851546 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.851524 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kv8xc\"" Jul 16 12:40:47.851730 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.851657 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 12:40:47.851949 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.851931 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 12:40:47.852232 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.852216 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 12:40:47.852686 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.852667 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 12:40:47.856717 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.856377 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-serving-cert\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.856717 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.856425 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx9x6\" (UniqueName: \"kubernetes.io/projected/b6fcd828-ebf4-440c-9b04-a7c679f444e8-kube-api-access-hx9x6\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.856717 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.856484 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-oauth-config\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.856717 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.856540 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-service-ca\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.856717 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.856607 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-config\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.856717 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.856636 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-oauth-serving-cert\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.856717 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.856653 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f5cd5fdbf-d7db9"] Jul 16 12:40:47.957185 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.957154 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-serving-cert\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.957185 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.957189 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx9x6\" (UniqueName: \"kubernetes.io/projected/b6fcd828-ebf4-440c-9b04-a7c679f444e8-kube-api-access-hx9x6\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.957383 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.957224 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-oauth-config\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.957383 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.957254 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-service-ca\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.957383 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.957284 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-config\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.957383 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.957304 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-oauth-serving-cert\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.957979 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.957950 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-service-ca\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.958117 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.957994 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-oauth-serving-cert\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.958117 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.958093 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-config\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.959661 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.959639 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-oauth-config\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.959762 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.959734 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-serving-cert\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:47.965605 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:47.965578 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx9x6\" (UniqueName: \"kubernetes.io/projected/b6fcd828-ebf4-440c-9b04-a7c679f444e8-kube-api-access-hx9x6\") pod \"console-6f5cd5fdbf-d7db9\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:48.158117 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:48.158017 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:48.287333 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:48.287311 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f5cd5fdbf-d7db9"] Jul 16 12:40:48.289645 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:40:48.289619 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6fcd828_ebf4_440c_9b04_a7c679f444e8.slice/crio-aca80141c4f6f10956c1c8d69adc498e6b466d327e88c00850d208a1c613e6b7 WatchSource:0}: Error finding container aca80141c4f6f10956c1c8d69adc498e6b466d327e88c00850d208a1c613e6b7: Status 404 returned error can't find the container with id aca80141c4f6f10956c1c8d69adc498e6b466d327e88c00850d208a1c613e6b7 Jul 16 12:40:49.082718 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:49.082687 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5cd5fdbf-d7db9" event={"ID":"b6fcd828-ebf4-440c-9b04-a7c679f444e8","Type":"ContainerStarted","Data":"aca80141c4f6f10956c1c8d69adc498e6b466d327e88c00850d208a1c613e6b7"} Jul 16 12:40:49.917856 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:49.917829 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sjhtn" Jul 16 12:40:52.097644 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:52.097606 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5cd5fdbf-d7db9" event={"ID":"b6fcd828-ebf4-440c-9b04-a7c679f444e8","Type":"ContainerStarted","Data":"6b9ff8dd38e2cae9ded2914f40869b26a8769b88c0cd54b0d69ca8a182473cc1"} Jul 16 12:40:52.117396 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:52.117345 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f5cd5fdbf-d7db9" podStartSLOduration=1.4680208000000001 podStartE2EDuration="5.117330793s" totalImagesPullingTime="3.649309993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:40:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:40:52.116779698 +0000 UTC m=+105.350687563" watchObservedRunningTime="2026-07-16 12:40:52.117330793 +0000 UTC m=+105.351238643" Jul 16 12:40:52.982462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:52.982435 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76d4b7d8fd-2b8r7" Jul 16 12:40:58.158485 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:58.158450 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:58.158940 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:58.158510 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:58.163182 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:58.163159 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:40:59.128940 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:40:59.128911 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:41:06.229226 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.229190 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6966c68947-mqrcb"] Jul 16 12:41:06.353212 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.353173 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6966c68947-mqrcb"] Jul 16 12:41:06.353391 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.353289 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.366426 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.366397 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 12:41:06.415137 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.415096 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-serving-cert\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.415307 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.415150 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-oauth-config\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.415307 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.415224 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-config\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.415307 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.415260 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-oauth-serving-cert\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.415307 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.415279 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9l5f\" (UniqueName: \"kubernetes.io/projected/6fd6fb9c-75af-4af6-b53c-f76f026ca642-kube-api-access-n9l5f\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.415441 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.415314 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-service-ca\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.415441 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.415364 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-trusted-ca-bundle\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.516580 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.516487 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-trusted-ca-bundle\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.516580 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.516545 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-serving-cert\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.516850 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.516587 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-oauth-config\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.516850 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.516626 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-config\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.516850 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.516673 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-oauth-serving-cert\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.516850 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.516697 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9l5f\" (UniqueName: \"kubernetes.io/projected/6fd6fb9c-75af-4af6-b53c-f76f026ca642-kube-api-access-n9l5f\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.516850 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.516718 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-service-ca\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.517437 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.517409 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-service-ca\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.517546 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.517409 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-oauth-serving-cert\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.517546 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.517484 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-config\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.517730 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.517706 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-trusted-ca-bundle\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.519119 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.519097 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-oauth-config\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.519270 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.519252 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-serving-cert\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.525722 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.525700 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9l5f\" (UniqueName: \"kubernetes.io/projected/6fd6fb9c-75af-4af6-b53c-f76f026ca642-kube-api-access-n9l5f\") pod \"console-6966c68947-mqrcb\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.668616 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.668580 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:06.825492 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:06.825453 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6966c68947-mqrcb"] Jul 16 12:41:06.828755 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:41:06.828720 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fd6fb9c_75af_4af6_b53c_f76f026ca642.slice/crio-611614aa926fd0ba570fd400a47306f013511c5a0975a021fb3d5d50f6e10d90 WatchSource:0}: Error finding container 611614aa926fd0ba570fd400a47306f013511c5a0975a021fb3d5d50f6e10d90: Status 404 returned error can't find the container with id 611614aa926fd0ba570fd400a47306f013511c5a0975a021fb3d5d50f6e10d90 Jul 16 12:41:07.156204 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:07.156126 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6966c68947-mqrcb" event={"ID":"6fd6fb9c-75af-4af6-b53c-f76f026ca642","Type":"ContainerStarted","Data":"708d6980ee23e2d03abadb535b28ae16151e4c80c0e40491cabe072fa5a39a30"} Jul 16 12:41:07.156204 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:07.156175 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6966c68947-mqrcb" event={"ID":"6fd6fb9c-75af-4af6-b53c-f76f026ca642","Type":"ContainerStarted","Data":"611614aa926fd0ba570fd400a47306f013511c5a0975a021fb3d5d50f6e10d90"} Jul 16 12:41:07.177672 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:07.177625 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6966c68947-mqrcb" podStartSLOduration=1.177609495 podStartE2EDuration="1.177609495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:41:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:41:07.176613556 +0000 UTC m=+120.410521408" watchObservedRunningTime="2026-07-16 12:41:07.177609495 +0000 UTC m=+120.411517345" Jul 16 12:41:16.669318 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:16.669277 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:16.669726 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:16.669341 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:16.674208 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:16.674187 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:17.105674 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:17.105633 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs\") pod \"network-metrics-daemon-d6kvg\" (UID: \"8892e9d1-ce2b-47d1-af07-27d1e94dd31c\") " pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:41:17.108028 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:17.108008 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8892e9d1-ce2b-47d1-af07-27d1e94dd31c-metrics-certs\") pod \"network-metrics-daemon-d6kvg\" (UID: \"8892e9d1-ce2b-47d1-af07-27d1e94dd31c\") " pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:41:17.197225 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:17.197198 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:41:17.250851 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:17.249604 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f5cd5fdbf-d7db9"] Jul 16 12:41:17.261282 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:17.261068 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8df2z\"" Jul 16 12:41:17.269414 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:17.269341 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6kvg" Jul 16 12:41:17.420893 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:17.420867 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d6kvg"] Jul 16 12:41:17.423725 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:41:17.423698 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8892e9d1_ce2b_47d1_af07_27d1e94dd31c.slice/crio-89e8f3c370e4d48552afe3e5e54cdac9dad8f3b24a69f2c5d4e62b534b02e062 WatchSource:0}: Error finding container 89e8f3c370e4d48552afe3e5e54cdac9dad8f3b24a69f2c5d4e62b534b02e062: Status 404 returned error can't find the container with id 89e8f3c370e4d48552afe3e5e54cdac9dad8f3b24a69f2c5d4e62b534b02e062 Jul 16 12:41:18.197719 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:18.197677 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d6kvg" event={"ID":"8892e9d1-ce2b-47d1-af07-27d1e94dd31c","Type":"ContainerStarted","Data":"89e8f3c370e4d48552afe3e5e54cdac9dad8f3b24a69f2c5d4e62b534b02e062"} Jul 16 12:41:20.207492 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:20.207460 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d6kvg" event={"ID":"8892e9d1-ce2b-47d1-af07-27d1e94dd31c","Type":"ContainerStarted","Data":"6593f1dce9ec08e584218b2c01a39557e00b387eacac34ff462c42e97e435c54"} Jul 16 12:41:21.212806 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:21.212772 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d6kvg" event={"ID":"8892e9d1-ce2b-47d1-af07-27d1e94dd31c","Type":"ContainerStarted","Data":"e67570fceac91283fc9a6354e9ed64d9596b9322bfda0636e28bd3b9115f7155"} Jul 16 12:41:21.233546 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:21.233494 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-d6kvg" podStartSLOduration=130.580808469 podStartE2EDuration="2m13.233478276s" totalImagesPullingTime="2.652669807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:41:21.232503672 +0000 UTC m=+134.466411505" watchObservedRunningTime="2026-07-16 12:41:21.233478276 +0000 UTC m=+134.467386384" Jul 16 12:41:42.284989 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.284941 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f5cd5fdbf-d7db9" podUID="b6fcd828-ebf4-440c-9b04-a7c679f444e8" containerName="console" containerID="cri-o://6b9ff8dd38e2cae9ded2914f40869b26a8769b88c0cd54b0d69ca8a182473cc1" gracePeriod=15 Jul 16 12:41:42.522687 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.522663 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:41:42.619737 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.619652 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-serving-cert\") pod \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " Jul 16 12:41:42.619737 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.619696 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-oauth-serving-cert\") pod \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " Jul 16 12:41:42.619964 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.619753 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-oauth-config\") pod \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " Jul 16 12:41:42.619964 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.619778 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-config\") pod \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " Jul 16 12:41:42.619964 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.619798 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx9x6\" (UniqueName: \"kubernetes.io/projected/b6fcd828-ebf4-440c-9b04-a7c679f444e8-kube-api-access-hx9x6\") pod \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " Jul 16 12:41:42.619964 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.619814 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-service-ca\") pod \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\" (UID: \"b6fcd828-ebf4-440c-9b04-a7c679f444e8\") " Jul 16 12:41:42.620267 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.620241 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b6fcd828-ebf4-440c-9b04-a7c679f444e8" (UID: "b6fcd828-ebf4-440c-9b04-a7c679f444e8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:41:42.620345 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.620312 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-service-ca" (OuterVolumeSpecName: "service-ca") pod "b6fcd828-ebf4-440c-9b04-a7c679f444e8" (UID: "b6fcd828-ebf4-440c-9b04-a7c679f444e8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:41:42.620387 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.620341 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-config" (OuterVolumeSpecName: "console-config") pod "b6fcd828-ebf4-440c-9b04-a7c679f444e8" (UID: "b6fcd828-ebf4-440c-9b04-a7c679f444e8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:41:42.622164 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.622133 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6fcd828-ebf4-440c-9b04-a7c679f444e8-kube-api-access-hx9x6" (OuterVolumeSpecName: "kube-api-access-hx9x6") pod "b6fcd828-ebf4-440c-9b04-a7c679f444e8" (UID: "b6fcd828-ebf4-440c-9b04-a7c679f444e8"). InnerVolumeSpecName "kube-api-access-hx9x6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:41:42.622271 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.622191 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b6fcd828-ebf4-440c-9b04-a7c679f444e8" (UID: "b6fcd828-ebf4-440c-9b04-a7c679f444e8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:41:42.622271 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.622218 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b6fcd828-ebf4-440c-9b04-a7c679f444e8" (UID: "b6fcd828-ebf4-440c-9b04-a7c679f444e8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:41:42.720507 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.720470 2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-oauth-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:41:42.720507 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.720499 2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:41:42.720507 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.720509 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx9x6\" (UniqueName: \"kubernetes.io/projected/b6fcd828-ebf4-440c-9b04-a7c679f444e8-kube-api-access-hx9x6\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:41:42.720507 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.720518 2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-service-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:41:42.720772 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.720526 2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6fcd828-ebf4-440c-9b04-a7c679f444e8-console-serving-cert\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:41:42.720772 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:42.720534 2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6fcd828-ebf4-440c-9b04-a7c679f444e8-oauth-serving-cert\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:41:43.303451 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:43.303414 2567 generic.go:356] "Generic (PLEG): container finished" podID="b6fcd828-ebf4-440c-9b04-a7c679f444e8" containerID="6b9ff8dd38e2cae9ded2914f40869b26a8769b88c0cd54b0d69ca8a182473cc1" exitCode=0 Jul 16 12:41:43.303838 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:43.303462 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5cd5fdbf-d7db9" event={"ID":"b6fcd828-ebf4-440c-9b04-a7c679f444e8","Type":"ContainerDied","Data":"6b9ff8dd38e2cae9ded2914f40869b26a8769b88c0cd54b0d69ca8a182473cc1"} Jul 16 12:41:43.303838 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:43.303467 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5cd5fdbf-d7db9" Jul 16 12:41:43.303838 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:43.303506 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5cd5fdbf-d7db9" event={"ID":"b6fcd828-ebf4-440c-9b04-a7c679f444e8","Type":"ContainerDied","Data":"aca80141c4f6f10956c1c8d69adc498e6b466d327e88c00850d208a1c613e6b7"} Jul 16 12:41:43.303838 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:43.303530 2567 scope.go:117] "RemoveContainer" containerID="6b9ff8dd38e2cae9ded2914f40869b26a8769b88c0cd54b0d69ca8a182473cc1" Jul 16 12:41:43.312641 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:43.312622 2567 scope.go:117] "RemoveContainer" containerID="6b9ff8dd38e2cae9ded2914f40869b26a8769b88c0cd54b0d69ca8a182473cc1" Jul 16 12:41:43.312898 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:41:43.312881 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b9ff8dd38e2cae9ded2914f40869b26a8769b88c0cd54b0d69ca8a182473cc1\": container with ID starting with 6b9ff8dd38e2cae9ded2914f40869b26a8769b88c0cd54b0d69ca8a182473cc1 not found: ID does not exist" containerID="6b9ff8dd38e2cae9ded2914f40869b26a8769b88c0cd54b0d69ca8a182473cc1" Jul 16 12:41:43.312947 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:43.312906 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b9ff8dd38e2cae9ded2914f40869b26a8769b88c0cd54b0d69ca8a182473cc1"} err="failed to get container status \"6b9ff8dd38e2cae9ded2914f40869b26a8769b88c0cd54b0d69ca8a182473cc1\": rpc error: code = NotFound desc = could not find container \"6b9ff8dd38e2cae9ded2914f40869b26a8769b88c0cd54b0d69ca8a182473cc1\": container with ID starting with 6b9ff8dd38e2cae9ded2914f40869b26a8769b88c0cd54b0d69ca8a182473cc1 not found: ID does not exist" Jul 16 12:41:43.323307 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:43.323282 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f5cd5fdbf-d7db9"] Jul 16 12:41:43.328820 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:43.328796 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f5cd5fdbf-d7db9"] Jul 16 12:41:43.638275 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:41:43.638192 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6fcd828-ebf4-440c-9b04-a7c679f444e8" path="/var/lib/kubelet/pods/b6fcd828-ebf4-440c-9b04-a7c679f444e8/volumes" Jul 16 12:42:23.803450 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:23.803415 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6966c68947-mqrcb"] Jul 16 12:42:48.825302 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:48.825249 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6966c68947-mqrcb" podUID="6fd6fb9c-75af-4af6-b53c-f76f026ca642" containerName="console" containerID="cri-o://708d6980ee23e2d03abadb535b28ae16151e4c80c0e40491cabe072fa5a39a30" gracePeriod=15 Jul 16 12:42:49.064217 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.064185 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:42:49.133396 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.133326 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-service-ca\") pod \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " Jul 16 12:42:49.133396 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.133374 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-trusted-ca-bundle\") pod \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " Jul 16 12:42:49.133396 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.133395 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-oauth-config\") pod \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " Jul 16 12:42:49.133626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.133524 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-config\") pod \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " Jul 16 12:42:49.133626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.133592 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-serving-cert\") pod \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " Jul 16 12:42:49.133626 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.133620 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-oauth-serving-cert\") pod \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " Jul 16 12:42:49.133775 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.133666 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9l5f\" (UniqueName: \"kubernetes.io/projected/6fd6fb9c-75af-4af6-b53c-f76f026ca642-kube-api-access-n9l5f\") pod \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\" (UID: \"6fd6fb9c-75af-4af6-b53c-f76f026ca642\") " Jul 16 12:42:49.133775 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.133751 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6fd6fb9c-75af-4af6-b53c-f76f026ca642" (UID: "6fd6fb9c-75af-4af6-b53c-f76f026ca642"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:42:49.133872 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.133767 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-service-ca" (OuterVolumeSpecName: "service-ca") pod "6fd6fb9c-75af-4af6-b53c-f76f026ca642" (UID: "6fd6fb9c-75af-4af6-b53c-f76f026ca642"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:42:49.133930 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.133827 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-config" (OuterVolumeSpecName: "console-config") pod "6fd6fb9c-75af-4af6-b53c-f76f026ca642" (UID: "6fd6fb9c-75af-4af6-b53c-f76f026ca642"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:42:49.134077 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.134017 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6fd6fb9c-75af-4af6-b53c-f76f026ca642" (UID: "6fd6fb9c-75af-4af6-b53c-f76f026ca642"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:42:49.134077 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.134067 2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:42:49.134234 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.134088 2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-service-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:42:49.134234 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.134101 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-trusted-ca-bundle\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:42:49.135557 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.135529 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6fd6fb9c-75af-4af6-b53c-f76f026ca642" (UID: "6fd6fb9c-75af-4af6-b53c-f76f026ca642"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:42:49.135670 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.135645 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fd6fb9c-75af-4af6-b53c-f76f026ca642-kube-api-access-n9l5f" (OuterVolumeSpecName: "kube-api-access-n9l5f") pod "6fd6fb9c-75af-4af6-b53c-f76f026ca642" (UID: "6fd6fb9c-75af-4af6-b53c-f76f026ca642"). InnerVolumeSpecName "kube-api-access-n9l5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:42:49.135711 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.135660 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6fd6fb9c-75af-4af6-b53c-f76f026ca642" (UID: "6fd6fb9c-75af-4af6-b53c-f76f026ca642"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:42:49.239017 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.234877 2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-serving-cert\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:42:49.239017 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.234915 2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fd6fb9c-75af-4af6-b53c-f76f026ca642-oauth-serving-cert\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:42:49.239017 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.234939 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9l5f\" (UniqueName: \"kubernetes.io/projected/6fd6fb9c-75af-4af6-b53c-f76f026ca642-kube-api-access-n9l5f\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:42:49.239017 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.234953 2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fd6fb9c-75af-4af6-b53c-f76f026ca642-console-oauth-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:42:49.540394 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.540360 2567 generic.go:356] "Generic (PLEG): container finished" podID="6fd6fb9c-75af-4af6-b53c-f76f026ca642" containerID="708d6980ee23e2d03abadb535b28ae16151e4c80c0e40491cabe072fa5a39a30" exitCode=0 Jul 16 12:42:49.540560 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.540415 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6966c68947-mqrcb" Jul 16 12:42:49.540560 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.540442 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6966c68947-mqrcb" event={"ID":"6fd6fb9c-75af-4af6-b53c-f76f026ca642","Type":"ContainerDied","Data":"708d6980ee23e2d03abadb535b28ae16151e4c80c0e40491cabe072fa5a39a30"} Jul 16 12:42:49.540560 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.540526 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6966c68947-mqrcb" event={"ID":"6fd6fb9c-75af-4af6-b53c-f76f026ca642","Type":"ContainerDied","Data":"611614aa926fd0ba570fd400a47306f013511c5a0975a021fb3d5d50f6e10d90"} Jul 16 12:42:49.540560 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.540551 2567 scope.go:117] "RemoveContainer" containerID="708d6980ee23e2d03abadb535b28ae16151e4c80c0e40491cabe072fa5a39a30" Jul 16 12:42:49.548697 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.548677 2567 scope.go:117] "RemoveContainer" containerID="708d6980ee23e2d03abadb535b28ae16151e4c80c0e40491cabe072fa5a39a30" Jul 16 12:42:49.548947 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:42:49.548922 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"708d6980ee23e2d03abadb535b28ae16151e4c80c0e40491cabe072fa5a39a30\": container with ID starting with 708d6980ee23e2d03abadb535b28ae16151e4c80c0e40491cabe072fa5a39a30 not found: ID does not exist" containerID="708d6980ee23e2d03abadb535b28ae16151e4c80c0e40491cabe072fa5a39a30" Jul 16 12:42:49.548991 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.548950 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"708d6980ee23e2d03abadb535b28ae16151e4c80c0e40491cabe072fa5a39a30"} err="failed to get container status \"708d6980ee23e2d03abadb535b28ae16151e4c80c0e40491cabe072fa5a39a30\": rpc error: code = NotFound desc = could not find container \"708d6980ee23e2d03abadb535b28ae16151e4c80c0e40491cabe072fa5a39a30\": container with ID starting with 708d6980ee23e2d03abadb535b28ae16151e4c80c0e40491cabe072fa5a39a30 not found: ID does not exist" Jul 16 12:42:49.560999 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.560976 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6966c68947-mqrcb"] Jul 16 12:42:49.564575 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.564553 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6966c68947-mqrcb"] Jul 16 12:42:49.637749 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:42:49.637725 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fd6fb9c-75af-4af6-b53c-f76f026ca642" path="/var/lib/kubelet/pods/6fd6fb9c-75af-4af6-b53c-f76f026ca642/volumes" Jul 16 12:43:02.742468 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.742433 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-97xsg"] Jul 16 12:43:02.742943 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.742886 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6fcd828-ebf4-440c-9b04-a7c679f444e8" containerName="console" Jul 16 12:43:02.742943 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.742907 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6fcd828-ebf4-440c-9b04-a7c679f444e8" containerName="console" Jul 16 12:43:02.742943 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.742924 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd6fb9c-75af-4af6-b53c-f76f026ca642" containerName="console" Jul 16 12:43:02.742943 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.742933 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd6fb9c-75af-4af6-b53c-f76f026ca642" containerName="console" Jul 16 12:43:02.743163 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.743002 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fd6fb9c-75af-4af6-b53c-f76f026ca642" containerName="console" Jul 16 12:43:02.743163 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.743012 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6fcd828-ebf4-440c-9b04-a7c679f444e8" containerName="console" Jul 16 12:43:02.745019 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.744993 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-97xsg" Jul 16 12:43:02.747571 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.747540 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 16 12:43:02.747571 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.747554 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-4t7sg\"" Jul 16 12:43:02.747728 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.747684 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 16 12:43:02.759728 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.759709 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-97xsg"] Jul 16 12:43:02.834027 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.833996 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/877f431b-e75e-403a-8222-29c1de6f42ba-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-97xsg\" (UID: \"877f431b-e75e-403a-8222-29c1de6f42ba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-97xsg" Jul 16 12:43:02.834178 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.834124 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br8vw\" (UniqueName: \"kubernetes.io/projected/877f431b-e75e-403a-8222-29c1de6f42ba-kube-api-access-br8vw\") pod \"cert-manager-operator-controller-manager-55c444bf86-97xsg\" (UID: \"877f431b-e75e-403a-8222-29c1de6f42ba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-97xsg" Jul 16 12:43:02.935085 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.935033 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br8vw\" (UniqueName: \"kubernetes.io/projected/877f431b-e75e-403a-8222-29c1de6f42ba-kube-api-access-br8vw\") pod \"cert-manager-operator-controller-manager-55c444bf86-97xsg\" (UID: \"877f431b-e75e-403a-8222-29c1de6f42ba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-97xsg" Jul 16 12:43:02.935206 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.935109 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/877f431b-e75e-403a-8222-29c1de6f42ba-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-97xsg\" (UID: \"877f431b-e75e-403a-8222-29c1de6f42ba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-97xsg" Jul 16 12:43:02.935458 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.935442 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/877f431b-e75e-403a-8222-29c1de6f42ba-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-97xsg\" (UID: \"877f431b-e75e-403a-8222-29c1de6f42ba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-97xsg" Jul 16 12:43:02.944257 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:02.944233 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br8vw\" (UniqueName: \"kubernetes.io/projected/877f431b-e75e-403a-8222-29c1de6f42ba-kube-api-access-br8vw\") pod \"cert-manager-operator-controller-manager-55c444bf86-97xsg\" (UID: \"877f431b-e75e-403a-8222-29c1de6f42ba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-97xsg" Jul 16 12:43:03.054335 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:03.054259 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-97xsg" Jul 16 12:43:03.183907 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:03.183876 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-97xsg"] Jul 16 12:43:03.187600 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:43:03.187568 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod877f431b_e75e_403a_8222_29c1de6f42ba.slice/crio-6e8ddd4e26b7e55106b38e0c294fadb01c9cb0e80a11a11f54501f1bdc47c15f WatchSource:0}: Error finding container 6e8ddd4e26b7e55106b38e0c294fadb01c9cb0e80a11a11f54501f1bdc47c15f: Status 404 returned error can't find the container with id 6e8ddd4e26b7e55106b38e0c294fadb01c9cb0e80a11a11f54501f1bdc47c15f Jul 16 12:43:03.597451 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:03.597416 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-97xsg" event={"ID":"877f431b-e75e-403a-8222-29c1de6f42ba","Type":"ContainerStarted","Data":"6e8ddd4e26b7e55106b38e0c294fadb01c9cb0e80a11a11f54501f1bdc47c15f"} Jul 16 12:43:06.610995 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:06.610963 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-97xsg" event={"ID":"877f431b-e75e-403a-8222-29c1de6f42ba","Type":"ContainerStarted","Data":"b82a5daf7d8d093102010b6c2fc6d8d39eacd0e6cc216b7b3cf91b81281b1659"} Jul 16 12:43:06.633843 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:06.633779 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-97xsg" podStartSLOduration=2.107022479 podStartE2EDuration="4.633761644s" totalImagesPullingTime="2.526739165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:43:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:43:06.631520286 +0000 UTC m=+239.865428138" watchObservedRunningTime="2026-07-16 12:43:06.633761644 +0000 UTC m=+239.867669495" Jul 16 12:43:07.899322 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:07.899283 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7mvr4"] Jul 16 12:43:07.903640 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:07.903618 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7mvr4" Jul 16 12:43:07.906364 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:07.906341 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 16 12:43:07.907805 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:07.907786 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-66mkw\"" Jul 16 12:43:07.907923 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:07.907807 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 16 12:43:07.911212 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:07.911191 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7mvr4"] Jul 16 12:43:07.979273 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:07.979241 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/566d1da6-f25b-499d-a72f-78645bcc71fc-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7mvr4\" (UID: \"566d1da6-f25b-499d-a72f-78645bcc71fc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7mvr4" Jul 16 12:43:07.979424 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:07.979284 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lghct\" (UniqueName: \"kubernetes.io/projected/566d1da6-f25b-499d-a72f-78645bcc71fc-kube-api-access-lghct\") pod \"cert-manager-webhook-6486ffd997-7mvr4\" (UID: \"566d1da6-f25b-499d-a72f-78645bcc71fc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7mvr4" Jul 16 12:43:08.080533 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:08.080494 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/566d1da6-f25b-499d-a72f-78645bcc71fc-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7mvr4\" (UID: \"566d1da6-f25b-499d-a72f-78645bcc71fc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7mvr4" Jul 16 12:43:08.080705 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:08.080547 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lghct\" (UniqueName: \"kubernetes.io/projected/566d1da6-f25b-499d-a72f-78645bcc71fc-kube-api-access-lghct\") pod \"cert-manager-webhook-6486ffd997-7mvr4\" (UID: \"566d1da6-f25b-499d-a72f-78645bcc71fc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7mvr4" Jul 16 12:43:08.090299 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:08.090268 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/566d1da6-f25b-499d-a72f-78645bcc71fc-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7mvr4\" (UID: \"566d1da6-f25b-499d-a72f-78645bcc71fc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7mvr4" Jul 16 12:43:08.090424 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:08.090352 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lghct\" (UniqueName: \"kubernetes.io/projected/566d1da6-f25b-499d-a72f-78645bcc71fc-kube-api-access-lghct\") pod \"cert-manager-webhook-6486ffd997-7mvr4\" (UID: \"566d1da6-f25b-499d-a72f-78645bcc71fc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7mvr4" Jul 16 12:43:08.226690 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:08.226658 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7mvr4" Jul 16 12:43:08.359897 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:08.359865 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7mvr4"] Jul 16 12:43:08.363228 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:43:08.363203 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod566d1da6_f25b_499d_a72f_78645bcc71fc.slice/crio-6d4a374362e288a1958c6652af2c80dbba7d326dc3449e8240103cb61574cd47 WatchSource:0}: Error finding container 6d4a374362e288a1958c6652af2c80dbba7d326dc3449e8240103cb61574cd47: Status 404 returned error can't find the container with id 6d4a374362e288a1958c6652af2c80dbba7d326dc3449e8240103cb61574cd47 Jul 16 12:43:08.620167 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:08.620084 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7mvr4" event={"ID":"566d1da6-f25b-499d-a72f-78645bcc71fc","Type":"ContainerStarted","Data":"6d4a374362e288a1958c6652af2c80dbba7d326dc3449e8240103cb61574cd47"} Jul 16 12:43:11.637843 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:11.637815 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7mvr4" Jul 16 12:43:11.638246 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:11.637852 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7mvr4" event={"ID":"566d1da6-f25b-499d-a72f-78645bcc71fc","Type":"ContainerStarted","Data":"0877f07cb31426cd8d6eb59a5d4a90072f404376d93a9420fa4bcd360bbe3928"} Jul 16 12:43:11.651673 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:11.651626 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-7mvr4" podStartSLOduration=2.23988547 podStartE2EDuration="4.651611214s" totalImagesPullingTime="2.411725744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:43:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:43:11.65043666 +0000 UTC m=+244.884344512" watchObservedRunningTime="2026-07-16 12:43:11.651611214 +0000 UTC m=+244.885519065" Jul 16 12:43:17.641783 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:17.641754 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7mvr4" Jul 16 12:43:44.383613 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.383536 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw"] Jul 16 12:43:44.388378 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.388356 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.396666 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.396645 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 12:43:44.396758 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.396659 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 12:43:44.397360 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.397338 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 12:43:44.397446 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.397351 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lqknb\"" Jul 16 12:43:44.397489 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.397447 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 12:43:44.428399 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.428368 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw"] Jul 16 12:43:44.484103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.484069 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce7229d5-45f5-49d3-90eb-5c3721cc44a8-tmp\") pod \"opendatahub-operator-controller-manager-65f8bdf75c-4dkrw\" (UID: \"ce7229d5-45f5-49d3-90eb-5c3721cc44a8\") " pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.484103 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.484102 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce7229d5-45f5-49d3-90eb-5c3721cc44a8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65f8bdf75c-4dkrw\" (UID: \"ce7229d5-45f5-49d3-90eb-5c3721cc44a8\") " pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.484333 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.484127 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce7229d5-45f5-49d3-90eb-5c3721cc44a8-webhook-cert\") pod \"opendatahub-operator-controller-manager-65f8bdf75c-4dkrw\" (UID: \"ce7229d5-45f5-49d3-90eb-5c3721cc44a8\") " pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.484333 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.484222 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc7vh\" (UniqueName: \"kubernetes.io/projected/ce7229d5-45f5-49d3-90eb-5c3721cc44a8-kube-api-access-vc7vh\") pod \"opendatahub-operator-controller-manager-65f8bdf75c-4dkrw\" (UID: \"ce7229d5-45f5-49d3-90eb-5c3721cc44a8\") " pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.484333 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.484284 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce7229d5-45f5-49d3-90eb-5c3721cc44a8-manifests\") pod \"opendatahub-operator-controller-manager-65f8bdf75c-4dkrw\" (UID: \"ce7229d5-45f5-49d3-90eb-5c3721cc44a8\") " pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.585217 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.585183 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc7vh\" (UniqueName: \"kubernetes.io/projected/ce7229d5-45f5-49d3-90eb-5c3721cc44a8-kube-api-access-vc7vh\") pod \"opendatahub-operator-controller-manager-65f8bdf75c-4dkrw\" (UID: \"ce7229d5-45f5-49d3-90eb-5c3721cc44a8\") " pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.585217 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.585225 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce7229d5-45f5-49d3-90eb-5c3721cc44a8-manifests\") pod \"opendatahub-operator-controller-manager-65f8bdf75c-4dkrw\" (UID: \"ce7229d5-45f5-49d3-90eb-5c3721cc44a8\") " pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.585470 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.585282 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce7229d5-45f5-49d3-90eb-5c3721cc44a8-tmp\") pod \"opendatahub-operator-controller-manager-65f8bdf75c-4dkrw\" (UID: \"ce7229d5-45f5-49d3-90eb-5c3721cc44a8\") " pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.585470 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.585299 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce7229d5-45f5-49d3-90eb-5c3721cc44a8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65f8bdf75c-4dkrw\" (UID: \"ce7229d5-45f5-49d3-90eb-5c3721cc44a8\") " pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.585470 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.585321 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce7229d5-45f5-49d3-90eb-5c3721cc44a8-webhook-cert\") pod \"opendatahub-operator-controller-manager-65f8bdf75c-4dkrw\" (UID: \"ce7229d5-45f5-49d3-90eb-5c3721cc44a8\") " pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.585737 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.585710 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce7229d5-45f5-49d3-90eb-5c3721cc44a8-manifests\") pod \"opendatahub-operator-controller-manager-65f8bdf75c-4dkrw\" (UID: \"ce7229d5-45f5-49d3-90eb-5c3721cc44a8\") " pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.587709 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.587684 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce7229d5-45f5-49d3-90eb-5c3721cc44a8-tmp\") pod \"opendatahub-operator-controller-manager-65f8bdf75c-4dkrw\" (UID: \"ce7229d5-45f5-49d3-90eb-5c3721cc44a8\") " pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.587817 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.587728 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce7229d5-45f5-49d3-90eb-5c3721cc44a8-webhook-cert\") pod \"opendatahub-operator-controller-manager-65f8bdf75c-4dkrw\" (UID: \"ce7229d5-45f5-49d3-90eb-5c3721cc44a8\") " pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.587817 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.587810 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce7229d5-45f5-49d3-90eb-5c3721cc44a8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65f8bdf75c-4dkrw\" (UID: \"ce7229d5-45f5-49d3-90eb-5c3721cc44a8\") " pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.592773 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.592751 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc7vh\" (UniqueName: \"kubernetes.io/projected/ce7229d5-45f5-49d3-90eb-5c3721cc44a8-kube-api-access-vc7vh\") pod \"opendatahub-operator-controller-manager-65f8bdf75c-4dkrw\" (UID: \"ce7229d5-45f5-49d3-90eb-5c3721cc44a8\") " pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.698662 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.698634 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:44.837031 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:44.836949 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw"] Jul 16 12:43:44.841228 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:43:44.841193 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce7229d5_45f5_49d3_90eb_5c3721cc44a8.slice/crio-2ef3ea9fd5d188c52bafbdb5204c2b730bfe1226f9c5a8c36ea427ce23365254 WatchSource:0}: Error finding container 2ef3ea9fd5d188c52bafbdb5204c2b730bfe1226f9c5a8c36ea427ce23365254: Status 404 returned error can't find the container with id 2ef3ea9fd5d188c52bafbdb5204c2b730bfe1226f9c5a8c36ea427ce23365254 Jul 16 12:43:45.768179 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:45.768138 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" event={"ID":"ce7229d5-45f5-49d3-90eb-5c3721cc44a8","Type":"ContainerStarted","Data":"2ef3ea9fd5d188c52bafbdb5204c2b730bfe1226f9c5a8c36ea427ce23365254"} Jul 16 12:43:48.782884 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:48.782847 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" event={"ID":"ce7229d5-45f5-49d3-90eb-5c3721cc44a8","Type":"ContainerStarted","Data":"468106169fcab00e7da8a9e00f6a817f4fc387a4de982070e23cbb88e9cd75d1"} Jul 16 12:43:49.788972 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:49.788928 2567 generic.go:356] "Generic (PLEG): container finished" podID="ce7229d5-45f5-49d3-90eb-5c3721cc44a8" containerID="468106169fcab00e7da8a9e00f6a817f4fc387a4de982070e23cbb88e9cd75d1" exitCode=0 Jul 16 12:43:49.789451 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:49.789003 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" event={"ID":"ce7229d5-45f5-49d3-90eb-5c3721cc44a8","Type":"ContainerDied","Data":"468106169fcab00e7da8a9e00f6a817f4fc387a4de982070e23cbb88e9cd75d1"} Jul 16 12:43:50.794805 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:50.794771 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" event={"ID":"ce7229d5-45f5-49d3-90eb-5c3721cc44a8","Type":"ContainerStarted","Data":"20980ba0b57cdcb14b7f84f3f2c411655b451585ed62ce031281f13997728fbf"} Jul 16 12:43:50.795218 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:50.794841 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:43:50.823936 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:43:50.823888 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" podStartSLOduration=1.993469421 podStartE2EDuration="6.823867545s" totalImagesPullingTime="3.750911158s" totalInitContainerRuntime="1.079486966s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:43:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:43:50.822765553 +0000 UTC m=+284.056673425" watchObservedRunningTime="2026-07-16 12:43:50.823867545 +0000 UTC m=+284.057775402" Jul 16 12:44:01.800326 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:01.800288 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65f8bdf75c-4dkrw" Jul 16 12:44:04.819995 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:04.819958 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6678559b48-jsd7l"] Jul 16 12:44:04.823198 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:04.823177 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6678559b48-jsd7l" Jul 16 12:44:04.825654 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:04.825633 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 16 12:44:04.825772 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:04.825712 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 16 12:44:04.825839 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:04.825822 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xmp2w\"" Jul 16 12:44:04.834272 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:04.834249 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6678559b48-jsd7l"] Jul 16 12:44:04.953391 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:04.953357 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0-tls-certs\") pod \"kube-auth-proxy-6678559b48-jsd7l\" (UID: \"dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0\") " pod="openshift-ingress/kube-auth-proxy-6678559b48-jsd7l" Jul 16 12:44:04.953557 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:04.953450 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bqb7\" (UniqueName: \"kubernetes.io/projected/dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0-kube-api-access-9bqb7\") pod \"kube-auth-proxy-6678559b48-jsd7l\" (UID: \"dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0\") " pod="openshift-ingress/kube-auth-proxy-6678559b48-jsd7l" Jul 16 12:44:04.953557 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:04.953483 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0-tmp\") pod \"kube-auth-proxy-6678559b48-jsd7l\" (UID: \"dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0\") " pod="openshift-ingress/kube-auth-proxy-6678559b48-jsd7l" Jul 16 12:44:05.054455 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.054418 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0-tls-certs\") pod \"kube-auth-proxy-6678559b48-jsd7l\" (UID: \"dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0\") " pod="openshift-ingress/kube-auth-proxy-6678559b48-jsd7l" Jul 16 12:44:05.054656 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.054490 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bqb7\" (UniqueName: \"kubernetes.io/projected/dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0-kube-api-access-9bqb7\") pod \"kube-auth-proxy-6678559b48-jsd7l\" (UID: \"dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0\") " pod="openshift-ingress/kube-auth-proxy-6678559b48-jsd7l" Jul 16 12:44:05.054656 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.054518 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0-tmp\") pod \"kube-auth-proxy-6678559b48-jsd7l\" (UID: \"dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0\") " pod="openshift-ingress/kube-auth-proxy-6678559b48-jsd7l" Jul 16 12:44:05.056840 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.056814 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0-tmp\") pod \"kube-auth-proxy-6678559b48-jsd7l\" (UID: \"dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0\") " pod="openshift-ingress/kube-auth-proxy-6678559b48-jsd7l" Jul 16 12:44:05.056969 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.056952 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0-tls-certs\") pod \"kube-auth-proxy-6678559b48-jsd7l\" (UID: \"dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0\") " pod="openshift-ingress/kube-auth-proxy-6678559b48-jsd7l" Jul 16 12:44:05.062739 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.062713 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bqb7\" (UniqueName: \"kubernetes.io/projected/dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0-kube-api-access-9bqb7\") pod \"kube-auth-proxy-6678559b48-jsd7l\" (UID: \"dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0\") " pod="openshift-ingress/kube-auth-proxy-6678559b48-jsd7l" Jul 16 12:44:05.132650 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.132557 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6678559b48-jsd7l" Jul 16 12:44:05.271136 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.271110 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6678559b48-jsd7l"] Jul 16 12:44:05.273819 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:44:05.273790 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddddc34cc_51e4_4a15_b5a3_2b2b4cb422c0.slice/crio-444e3e362968561bf8952ea98be360e4aef2927ceeb8a2cbaf05828dec92becf WatchSource:0}: Error finding container 444e3e362968561bf8952ea98be360e4aef2927ceeb8a2cbaf05828dec92becf: Status 404 returned error can't find the container with id 444e3e362968561bf8952ea98be360e4aef2927ceeb8a2cbaf05828dec92becf Jul 16 12:44:05.601167 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.601126 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gbk4m"] Jul 16 12:44:05.605383 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.605360 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gbk4m" Jul 16 12:44:05.608650 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.608623 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-wdsgb\"" Jul 16 12:44:05.613411 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.613390 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 16 12:44:05.627701 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.627672 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gbk4m"] Jul 16 12:44:05.661152 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.661119 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f28df006-ac3a-485b-860a-bbab526b2a9c-tls-certs\") pod \"model-serving-api-54d89bd5fc-gbk4m\" (UID: \"f28df006-ac3a-485b-860a-bbab526b2a9c\") " pod="opendatahub/model-serving-api-54d89bd5fc-gbk4m" Jul 16 12:44:05.661300 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.661226 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwc5k\" (UniqueName: \"kubernetes.io/projected/f28df006-ac3a-485b-860a-bbab526b2a9c-kube-api-access-lwc5k\") pod \"model-serving-api-54d89bd5fc-gbk4m\" (UID: \"f28df006-ac3a-485b-860a-bbab526b2a9c\") " pod="opendatahub/model-serving-api-54d89bd5fc-gbk4m" Jul 16 12:44:05.761738 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.761699 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f28df006-ac3a-485b-860a-bbab526b2a9c-tls-certs\") pod \"model-serving-api-54d89bd5fc-gbk4m\" (UID: \"f28df006-ac3a-485b-860a-bbab526b2a9c\") " pod="opendatahub/model-serving-api-54d89bd5fc-gbk4m" Jul 16 12:44:05.761946 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.761786 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwc5k\" (UniqueName: \"kubernetes.io/projected/f28df006-ac3a-485b-860a-bbab526b2a9c-kube-api-access-lwc5k\") pod \"model-serving-api-54d89bd5fc-gbk4m\" (UID: \"f28df006-ac3a-485b-860a-bbab526b2a9c\") " pod="opendatahub/model-serving-api-54d89bd5fc-gbk4m" Jul 16 12:44:05.764275 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.764236 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f28df006-ac3a-485b-860a-bbab526b2a9c-tls-certs\") pod \"model-serving-api-54d89bd5fc-gbk4m\" (UID: \"f28df006-ac3a-485b-860a-bbab526b2a9c\") " pod="opendatahub/model-serving-api-54d89bd5fc-gbk4m" Jul 16 12:44:05.772792 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.772760 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwc5k\" (UniqueName: \"kubernetes.io/projected/f28df006-ac3a-485b-860a-bbab526b2a9c-kube-api-access-lwc5k\") pod \"model-serving-api-54d89bd5fc-gbk4m\" (UID: \"f28df006-ac3a-485b-860a-bbab526b2a9c\") " pod="opendatahub/model-serving-api-54d89bd5fc-gbk4m" Jul 16 12:44:05.857188 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.857095 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6678559b48-jsd7l" event={"ID":"dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0","Type":"ContainerStarted","Data":"444e3e362968561bf8952ea98be360e4aef2927ceeb8a2cbaf05828dec92becf"} Jul 16 12:44:05.915547 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:05.915508 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gbk4m" Jul 16 12:44:06.068066 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.067901 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gbk4m"] Jul 16 12:44:06.070879 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:44:06.070843 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf28df006_ac3a_485b_860a_bbab526b2a9c.slice/crio-55bd57ccbbd1da3cb48bbe895960df2ef12429fee7ba6fde205eaeac8f65e67f WatchSource:0}: Error finding container 55bd57ccbbd1da3cb48bbe895960df2ef12429fee7ba6fde205eaeac8f65e67f: Status 404 returned error can't find the container with id 55bd57ccbbd1da3cb48bbe895960df2ef12429fee7ba6fde205eaeac8f65e67f Jul 16 12:44:06.197225 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.197193 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk"] Jul 16 12:44:06.200888 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.200855 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.203685 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.203655 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 16 12:44:06.203685 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.203655 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 16 12:44:06.204122 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.204013 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 16 12:44:06.204122 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.204099 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 12:44:06.204287 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.204102 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cp5wj\"" Jul 16 12:44:06.217079 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.217038 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk"] Jul 16 12:44:06.265734 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.265696 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-local-certs\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.265920 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.265762 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-cacerts\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.265920 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.265798 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9w2d\" (UniqueName: \"kubernetes.io/projected/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-kube-api-access-t9w2d\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.265920 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.265834 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-token\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.266155 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.265944 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.266155 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.265997 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.266155 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.266101 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.367374 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.367329 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-local-certs\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.367564 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.367400 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-cacerts\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.367564 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.367431 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9w2d\" (UniqueName: \"kubernetes.io/projected/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-kube-api-access-t9w2d\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.367564 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.367462 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-token\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.367741 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.367624 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.367741 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.367678 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.367741 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.367716 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.368420 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.368368 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.370286 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.370258 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-cacerts\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.370719 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.370698 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-local-certs\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.370814 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.370719 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.370876 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.370826 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.375799 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.375775 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-token\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.376241 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.376219 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9w2d\" (UniqueName: \"kubernetes.io/projected/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-kube-api-access-t9w2d\") pod \"istiod-openshift-gateway-7574459656-hb7tk\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.514999 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.514910 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:06.651560 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.651523 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk"] Jul 16 12:44:06.654930 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:44:06.654854 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69dcb810_0bb7_4b6b_9ee6_f1b1914965c0.slice/crio-07c24b175712336fb49f278ebb698246750d6dbb8fcf96977f42f3a173c26312 WatchSource:0}: Error finding container 07c24b175712336fb49f278ebb698246750d6dbb8fcf96977f42f3a173c26312: Status 404 returned error can't find the container with id 07c24b175712336fb49f278ebb698246750d6dbb8fcf96977f42f3a173c26312 Jul 16 12:44:06.864501 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.864414 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" event={"ID":"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0","Type":"ContainerStarted","Data":"07c24b175712336fb49f278ebb698246750d6dbb8fcf96977f42f3a173c26312"} Jul 16 12:44:06.866274 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:06.866234 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gbk4m" event={"ID":"f28df006-ac3a-485b-860a-bbab526b2a9c","Type":"ContainerStarted","Data":"55bd57ccbbd1da3cb48bbe895960df2ef12429fee7ba6fde205eaeac8f65e67f"} Jul 16 12:44:07.147911 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:07.147816 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 12:44:07.151468 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:07.151422 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 12:44:07.152343 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:07.152317 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 12:44:07.155954 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:07.155931 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 12:44:07.161342 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:07.161314 2567 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 12:44:09.644535 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:09.644503 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 16 12:44:09.652091 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:09.652060 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 12:44:09.652166 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:09.652122 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 12:44:09.884198 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:09.884151 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gbk4m" event={"ID":"f28df006-ac3a-485b-860a-bbab526b2a9c","Type":"ContainerStarted","Data":"0fe9ab7323ac92555bffbcf5e0186cb5c2ed7db4770e40a17fa0db53ac865244"} Jul 16 12:44:09.884665 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:09.884356 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-gbk4m" Jul 16 12:44:09.885966 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:09.885933 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" event={"ID":"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0","Type":"ContainerStarted","Data":"2645ef7ac7f743baa74df0350745a2c856b19d1fddd533dae49c7abb089d0878"} Jul 16 12:44:09.886115 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:09.885982 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:09.887359 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:09.887335 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6678559b48-jsd7l" event={"ID":"dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0","Type":"ContainerStarted","Data":"5ecec31b938ba60a6e94f1e0a2264b089db420923b6f4b16579b6fda55608612"} Jul 16 12:44:09.904016 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:09.903920 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-gbk4m" podStartSLOduration=1.331528576 podStartE2EDuration="4.903907164s" totalImagesPullingTime="3.572378588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:44:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:44:09.902415704 +0000 UTC m=+303.136323557" watchObservedRunningTime="2026-07-16 12:44:09.903907164 +0000 UTC m=+303.137815014" Jul 16 12:44:09.930649 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:09.930600 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" podStartSLOduration=0.936255736 podStartE2EDuration="3.930585845s" totalImagesPullingTime="2.994330109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:44:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:44:09.929289122 +0000 UTC m=+303.163196974" watchObservedRunningTime="2026-07-16 12:44:09.930585845 +0000 UTC m=+303.164493697" Jul 16 12:44:09.949631 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:09.949590 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6678559b48-jsd7l" podStartSLOduration=1.583347268 podStartE2EDuration="5.949576589s" totalImagesPullingTime="4.366229321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:44:09.948605525 +0000 UTC m=+303.182513378" watchObservedRunningTime="2026-07-16 12:44:09.949576589 +0000 UTC m=+303.183484439" Jul 16 12:44:10.893208 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:10.893175 2567 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-hb7tk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 12:44:10.893565 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:10.893240 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" podUID="69dcb810-0bb7-4b6b-9ee6-f1b1914965c0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 12:44:13.892545 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:13.892510 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:14.443779 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.443740 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj"] Jul 16 12:44:14.447475 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.447450 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj" Jul 16 12:44:14.449983 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.449961 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 16 12:44:14.450119 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.450086 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-mpshd\"" Jul 16 12:44:14.459207 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.459186 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6sqdk"] Jul 16 12:44:14.462308 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.462290 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6sqdk" Jul 16 12:44:14.465481 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.465456 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj"] Jul 16 12:44:14.465812 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.465794 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 16 12:44:14.465937 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.465819 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-cpq5q\"" Jul 16 12:44:14.483897 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.483868 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6sqdk"] Jul 16 12:44:14.550111 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.550081 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nhpn\" (UniqueName: \"kubernetes.io/projected/1c2296a3-023d-4d98-be47-2aa0bebb1f4f-kube-api-access-7nhpn\") pod \"llmisvc-controller-manager-777c6bd7d5-6sqdk\" (UID: \"1c2296a3-023d-4d98-be47-2aa0bebb1f4f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6sqdk" Jul 16 12:44:14.550272 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.550121 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh746\" (UniqueName: \"kubernetes.io/projected/674f92e7-2580-41cb-bf8a-5e735af66124-kube-api-access-gh746\") pod \"kserve-controller-manager-d6f4b65b8-wrhvj\" (UID: \"674f92e7-2580-41cb-bf8a-5e735af66124\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj" Jul 16 12:44:14.550272 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.550213 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/674f92e7-2580-41cb-bf8a-5e735af66124-cert\") pod \"kserve-controller-manager-d6f4b65b8-wrhvj\" (UID: \"674f92e7-2580-41cb-bf8a-5e735af66124\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj" Jul 16 12:44:14.550343 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.550271 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c2296a3-023d-4d98-be47-2aa0bebb1f4f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6sqdk\" (UID: \"1c2296a3-023d-4d98-be47-2aa0bebb1f4f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6sqdk" Jul 16 12:44:14.651298 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.651260 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/674f92e7-2580-41cb-bf8a-5e735af66124-cert\") pod \"kserve-controller-manager-d6f4b65b8-wrhvj\" (UID: \"674f92e7-2580-41cb-bf8a-5e735af66124\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj" Jul 16 12:44:14.651471 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.651307 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c2296a3-023d-4d98-be47-2aa0bebb1f4f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6sqdk\" (UID: \"1c2296a3-023d-4d98-be47-2aa0bebb1f4f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6sqdk" Jul 16 12:44:14.651471 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.651361 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nhpn\" (UniqueName: \"kubernetes.io/projected/1c2296a3-023d-4d98-be47-2aa0bebb1f4f-kube-api-access-7nhpn\") pod \"llmisvc-controller-manager-777c6bd7d5-6sqdk\" (UID: \"1c2296a3-023d-4d98-be47-2aa0bebb1f4f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6sqdk" Jul 16 12:44:14.651471 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.651395 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh746\" (UniqueName: \"kubernetes.io/projected/674f92e7-2580-41cb-bf8a-5e735af66124-kube-api-access-gh746\") pod \"kserve-controller-manager-d6f4b65b8-wrhvj\" (UID: \"674f92e7-2580-41cb-bf8a-5e735af66124\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj" Jul 16 12:44:14.651471 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:44:14.651420 2567 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 16 12:44:14.651605 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:44:14.651488 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/674f92e7-2580-41cb-bf8a-5e735af66124-cert podName:674f92e7-2580-41cb-bf8a-5e735af66124 nodeName:}" failed. No retries permitted until 2026-07-16 12:44:15.151467088 +0000 UTC m=+308.385374941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/674f92e7-2580-41cb-bf8a-5e735af66124-cert") pod "kserve-controller-manager-d6f4b65b8-wrhvj" (UID: "674f92e7-2580-41cb-bf8a-5e735af66124") : secret "kserve-webhook-server-cert" not found Jul 16 12:44:14.653945 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.653920 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c2296a3-023d-4d98-be47-2aa0bebb1f4f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6sqdk\" (UID: \"1c2296a3-023d-4d98-be47-2aa0bebb1f4f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6sqdk" Jul 16 12:44:14.677407 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.677369 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nhpn\" (UniqueName: \"kubernetes.io/projected/1c2296a3-023d-4d98-be47-2aa0bebb1f4f-kube-api-access-7nhpn\") pod \"llmisvc-controller-manager-777c6bd7d5-6sqdk\" (UID: \"1c2296a3-023d-4d98-be47-2aa0bebb1f4f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6sqdk" Jul 16 12:44:14.677882 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.677859 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh746\" (UniqueName: \"kubernetes.io/projected/674f92e7-2580-41cb-bf8a-5e735af66124-kube-api-access-gh746\") pod \"kserve-controller-manager-d6f4b65b8-wrhvj\" (UID: \"674f92e7-2580-41cb-bf8a-5e735af66124\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj" Jul 16 12:44:14.772067 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.771951 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6sqdk" Jul 16 12:44:14.918153 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.918122 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6sqdk"] Jul 16 12:44:14.920007 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:44:14.919973 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1c2296a3_023d_4d98_be47_2aa0bebb1f4f.slice/crio-7434440638a5efdd1575a60f32b0e4150616df502efcb7994b588b46deb22291 WatchSource:0}: Error finding container 7434440638a5efdd1575a60f32b0e4150616df502efcb7994b588b46deb22291: Status 404 returned error can't find the container with id 7434440638a5efdd1575a60f32b0e4150616df502efcb7994b588b46deb22291 Jul 16 12:44:14.921368 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:14.921345 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 12:44:15.157139 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:15.157013 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/674f92e7-2580-41cb-bf8a-5e735af66124-cert\") pod \"kserve-controller-manager-d6f4b65b8-wrhvj\" (UID: \"674f92e7-2580-41cb-bf8a-5e735af66124\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj" Jul 16 12:44:15.159438 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:15.159419 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/674f92e7-2580-41cb-bf8a-5e735af66124-cert\") pod \"kserve-controller-manager-d6f4b65b8-wrhvj\" (UID: \"674f92e7-2580-41cb-bf8a-5e735af66124\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj" Jul 16 12:44:15.357358 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:15.357327 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj" Jul 16 12:44:15.489217 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:15.489190 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj"] Jul 16 12:44:15.491465 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:44:15.491435 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod674f92e7_2580_41cb_bf8a_5e735af66124.slice/crio-084b95753cbe5170711718637893534dff6f66d1f6af05f1a9f5068ddfa1288b WatchSource:0}: Error finding container 084b95753cbe5170711718637893534dff6f66d1f6af05f1a9f5068ddfa1288b: Status 404 returned error can't find the container with id 084b95753cbe5170711718637893534dff6f66d1f6af05f1a9f5068ddfa1288b Jul 16 12:44:15.913426 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:15.913392 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6sqdk" event={"ID":"1c2296a3-023d-4d98-be47-2aa0bebb1f4f","Type":"ContainerStarted","Data":"7434440638a5efdd1575a60f32b0e4150616df502efcb7994b588b46deb22291"} Jul 16 12:44:15.914487 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:15.914460 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj" event={"ID":"674f92e7-2580-41cb-bf8a-5e735af66124","Type":"ContainerStarted","Data":"084b95753cbe5170711718637893534dff6f66d1f6af05f1a9f5068ddfa1288b"} Jul 16 12:44:18.619009 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.618966 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk"] Jul 16 12:44:18.619465 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.619271 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" podUID="69dcb810-0bb7-4b6b-9ee6-f1b1914965c0" containerName="discovery" containerID="cri-o://2645ef7ac7f743baa74df0350745a2c856b19d1fddd533dae49c7abb089d0878" gracePeriod=30 Jul 16 12:44:18.868949 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.868926 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:18.932999 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.932969 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6sqdk" event={"ID":"1c2296a3-023d-4d98-be47-2aa0bebb1f4f","Type":"ContainerStarted","Data":"4b6757355fd5ffea3963ea8426ee8e9bf378c60ce1d166a8a0627c61ab4a18a5"} Jul 16 12:44:18.933212 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.933087 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6sqdk" Jul 16 12:44:18.934549 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.934519 2567 generic.go:356] "Generic (PLEG): container finished" podID="69dcb810-0bb7-4b6b-9ee6-f1b1914965c0" containerID="2645ef7ac7f743baa74df0350745a2c856b19d1fddd533dae49c7abb089d0878" exitCode=0 Jul 16 12:44:18.934549 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.934546 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" Jul 16 12:44:18.934730 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.934591 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" event={"ID":"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0","Type":"ContainerDied","Data":"2645ef7ac7f743baa74df0350745a2c856b19d1fddd533dae49c7abb089d0878"} Jul 16 12:44:18.934730 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.934632 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk" event={"ID":"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0","Type":"ContainerDied","Data":"07c24b175712336fb49f278ebb698246750d6dbb8fcf96977f42f3a173c26312"} Jul 16 12:44:18.934730 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.934660 2567 scope.go:117] "RemoveContainer" containerID="2645ef7ac7f743baa74df0350745a2c856b19d1fddd533dae49c7abb089d0878" Jul 16 12:44:18.936294 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.936245 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj" event={"ID":"674f92e7-2580-41cb-bf8a-5e735af66124","Type":"ContainerStarted","Data":"0b5769af76590a2c89452935777584260e213dbf7acd75d82263bfcee060e0fa"} Jul 16 12:44:18.936401 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.936367 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj" Jul 16 12:44:18.942926 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.942907 2567 scope.go:117] "RemoveContainer" containerID="2645ef7ac7f743baa74df0350745a2c856b19d1fddd533dae49c7abb089d0878" Jul 16 12:44:18.943225 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:44:18.943197 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2645ef7ac7f743baa74df0350745a2c856b19d1fddd533dae49c7abb089d0878\": container with ID starting with 2645ef7ac7f743baa74df0350745a2c856b19d1fddd533dae49c7abb089d0878 not found: ID does not exist" containerID="2645ef7ac7f743baa74df0350745a2c856b19d1fddd533dae49c7abb089d0878" Jul 16 12:44:18.943298 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.943235 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2645ef7ac7f743baa74df0350745a2c856b19d1fddd533dae49c7abb089d0878"} err="failed to get container status \"2645ef7ac7f743baa74df0350745a2c856b19d1fddd533dae49c7abb089d0878\": rpc error: code = NotFound desc = could not find container \"2645ef7ac7f743baa74df0350745a2c856b19d1fddd533dae49c7abb089d0878\": container with ID starting with 2645ef7ac7f743baa74df0350745a2c856b19d1fddd533dae49c7abb089d0878 not found: ID does not exist" Jul 16 12:44:18.953211 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.953170 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6sqdk" podStartSLOduration=1.435756332 podStartE2EDuration="4.953157959s" totalImagesPullingTime="3.517401627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:44:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:44:18.951936795 +0000 UTC m=+312.185844647" watchObservedRunningTime="2026-07-16 12:44:18.953157959 +0000 UTC m=+312.187065949" Jul 16 12:44:18.969624 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.969587 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj" podStartSLOduration=2.021994445 podStartE2EDuration="4.969575191s" totalImagesPullingTime="2.947580746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:44:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:44:18.967921005 +0000 UTC m=+312.201828861" watchObservedRunningTime="2026-07-16 12:44:18.969575191 +0000 UTC m=+312.203483041" Jul 16 12:44:18.989194 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.989147 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-local-certs\") pod \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " Jul 16 12:44:18.989194 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.989174 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-cacerts\") pod \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " Jul 16 12:44:18.989319 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.989199 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-csr-dns-cert\") pod \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " Jul 16 12:44:18.989319 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.989245 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-kubeconfig\") pod \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " Jul 16 12:44:18.989319 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.989275 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9w2d\" (UniqueName: \"kubernetes.io/projected/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-kube-api-access-t9w2d\") pod \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " Jul 16 12:44:18.989319 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.989315 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-csr-ca-configmap\") pod \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " Jul 16 12:44:18.989509 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.989346 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-token\") pod \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\" (UID: \"69dcb810-0bb7-4b6b-9ee6-f1b1914965c0\") " Jul 16 12:44:18.989742 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.989709 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "69dcb810-0bb7-4b6b-9ee6-f1b1914965c0" (UID: "69dcb810-0bb7-4b6b-9ee6-f1b1914965c0"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:44:18.991489 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.991460 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-local-certs" (OuterVolumeSpecName: "local-certs") pod "69dcb810-0bb7-4b6b-9ee6-f1b1914965c0" (UID: "69dcb810-0bb7-4b6b-9ee6-f1b1914965c0"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:44:18.991569 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.991514 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-kube-api-access-t9w2d" (OuterVolumeSpecName: "kube-api-access-t9w2d") pod "69dcb810-0bb7-4b6b-9ee6-f1b1914965c0" (UID: "69dcb810-0bb7-4b6b-9ee6-f1b1914965c0"). InnerVolumeSpecName "kube-api-access-t9w2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:44:18.991651 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.991633 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-cacerts" (OuterVolumeSpecName: "cacerts") pod "69dcb810-0bb7-4b6b-9ee6-f1b1914965c0" (UID: "69dcb810-0bb7-4b6b-9ee6-f1b1914965c0"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:44:18.991717 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.991700 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "69dcb810-0bb7-4b6b-9ee6-f1b1914965c0" (UID: "69dcb810-0bb7-4b6b-9ee6-f1b1914965c0"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:44:18.991869 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.991849 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "69dcb810-0bb7-4b6b-9ee6-f1b1914965c0" (UID: "69dcb810-0bb7-4b6b-9ee6-f1b1914965c0"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:44:18.991922 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:18.991868 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "69dcb810-0bb7-4b6b-9ee6-f1b1914965c0" (UID: "69dcb810-0bb7-4b6b-9ee6-f1b1914965c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:44:19.090322 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:19.090292 2567 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-local-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:44:19.090322 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:19.090319 2567 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-cacerts\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:44:19.090525 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:19.090332 2567 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-csr-dns-cert\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:44:19.090525 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:19.090343 2567 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-kubeconfig\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:44:19.090525 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:19.090356 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9w2d\" (UniqueName: \"kubernetes.io/projected/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-kube-api-access-t9w2d\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:44:19.090525 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:19.090371 2567 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-csr-ca-configmap\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:44:19.090525 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:19.090383 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0-istio-token\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:44:19.256294 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:19.256252 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk"] Jul 16 12:44:19.260037 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:19.260013 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hb7tk"] Jul 16 12:44:19.639473 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:19.639396 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69dcb810-0bb7-4b6b-9ee6-f1b1914965c0" path="/var/lib/kubelet/pods/69dcb810-0bb7-4b6b-9ee6-f1b1914965c0/volumes" Jul 16 12:44:20.894811 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:20.894779 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-gbk4m" Jul 16 12:44:49.943637 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:49.943604 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6sqdk" Jul 16 12:44:49.946432 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:44:49.946415 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wrhvj" Jul 16 12:45:05.615992 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:05.615917 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-22ksl"] Jul 16 12:45:05.616437 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:05.616292 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69dcb810-0bb7-4b6b-9ee6-f1b1914965c0" containerName="discovery" Jul 16 12:45:05.616437 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:05.616307 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="69dcb810-0bb7-4b6b-9ee6-f1b1914965c0" containerName="discovery" Jul 16 12:45:05.616437 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:05.616411 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="69dcb810-0bb7-4b6b-9ee6-f1b1914965c0" containerName="discovery" Jul 16 12:45:05.619223 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:05.619204 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-22ksl" Jul 16 12:45:05.621721 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:05.621698 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 12:45:05.622931 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:05.622914 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 12:45:05.622993 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:05.622970 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-b265k\"" Jul 16 12:45:05.630368 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:05.630345 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-22ksl"] Jul 16 12:45:05.667773 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:05.667745 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l4fh\" (UniqueName: \"kubernetes.io/projected/77529ff5-36ad-4040-8ec3-8a3bc3c01c94-kube-api-access-8l4fh\") pod \"kuadrant-operator-catalog-22ksl\" (UID: \"77529ff5-36ad-4040-8ec3-8a3bc3c01c94\") " pod="kuadrant-system/kuadrant-operator-catalog-22ksl" Jul 16 12:45:05.768362 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:05.768327 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l4fh\" (UniqueName: \"kubernetes.io/projected/77529ff5-36ad-4040-8ec3-8a3bc3c01c94-kube-api-access-8l4fh\") pod \"kuadrant-operator-catalog-22ksl\" (UID: \"77529ff5-36ad-4040-8ec3-8a3bc3c01c94\") " pod="kuadrant-system/kuadrant-operator-catalog-22ksl" Jul 16 12:45:05.776378 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:05.776353 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l4fh\" (UniqueName: \"kubernetes.io/projected/77529ff5-36ad-4040-8ec3-8a3bc3c01c94-kube-api-access-8l4fh\") pod \"kuadrant-operator-catalog-22ksl\" (UID: \"77529ff5-36ad-4040-8ec3-8a3bc3c01c94\") " pod="kuadrant-system/kuadrant-operator-catalog-22ksl" Jul 16 12:45:05.929566 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:05.929545 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-22ksl" Jul 16 12:45:05.984280 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:05.984250 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-22ksl"] Jul 16 12:45:06.057187 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:06.057133 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-22ksl"] Jul 16 12:45:06.060003 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:45:06.059977 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77529ff5_36ad_4040_8ec3_8a3bc3c01c94.slice/crio-18c35b7eef6493ee7decf3e6c4e6fdb3391e9d41f165cebc013cc054b34e6703 WatchSource:0}: Error finding container 18c35b7eef6493ee7decf3e6c4e6fdb3391e9d41f165cebc013cc054b34e6703: Status 404 returned error can't find the container with id 18c35b7eef6493ee7decf3e6c4e6fdb3391e9d41f165cebc013cc054b34e6703 Jul 16 12:45:06.139882 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:06.139849 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-22ksl" event={"ID":"77529ff5-36ad-4040-8ec3-8a3bc3c01c94","Type":"ContainerStarted","Data":"18c35b7eef6493ee7decf3e6c4e6fdb3391e9d41f165cebc013cc054b34e6703"} Jul 16 12:45:06.194737 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:06.194664 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xp6gr"] Jul 16 12:45:06.198887 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:06.198869 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xp6gr" Jul 16 12:45:06.209397 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:06.209377 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xp6gr"] Jul 16 12:45:06.272694 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:06.272668 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdmw2\" (UniqueName: \"kubernetes.io/projected/4d4d0198-ddd2-4fbe-a81b-a9ac12923570-kube-api-access-hdmw2\") pod \"kuadrant-operator-catalog-xp6gr\" (UID: \"4d4d0198-ddd2-4fbe-a81b-a9ac12923570\") " pod="kuadrant-system/kuadrant-operator-catalog-xp6gr" Jul 16 12:45:06.373988 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:06.373953 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdmw2\" (UniqueName: \"kubernetes.io/projected/4d4d0198-ddd2-4fbe-a81b-a9ac12923570-kube-api-access-hdmw2\") pod \"kuadrant-operator-catalog-xp6gr\" (UID: \"4d4d0198-ddd2-4fbe-a81b-a9ac12923570\") " pod="kuadrant-system/kuadrant-operator-catalog-xp6gr" Jul 16 12:45:06.381576 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:06.381554 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdmw2\" (UniqueName: \"kubernetes.io/projected/4d4d0198-ddd2-4fbe-a81b-a9ac12923570-kube-api-access-hdmw2\") pod \"kuadrant-operator-catalog-xp6gr\" (UID: \"4d4d0198-ddd2-4fbe-a81b-a9ac12923570\") " pod="kuadrant-system/kuadrant-operator-catalog-xp6gr" Jul 16 12:45:06.508725 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:06.508645 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xp6gr" Jul 16 12:45:06.639405 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:06.639381 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xp6gr"] Jul 16 12:45:06.641930 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:45:06.641905 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d4d0198_ddd2_4fbe_a81b_a9ac12923570.slice/crio-adb25de3ca24e22eb1093776df904f87ca7ee415a620e96a183fb55e20bd1e78 WatchSource:0}: Error finding container adb25de3ca24e22eb1093776df904f87ca7ee415a620e96a183fb55e20bd1e78: Status 404 returned error can't find the container with id adb25de3ca24e22eb1093776df904f87ca7ee415a620e96a183fb55e20bd1e78 Jul 16 12:45:07.145343 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:07.145296 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xp6gr" event={"ID":"4d4d0198-ddd2-4fbe-a81b-a9ac12923570","Type":"ContainerStarted","Data":"adb25de3ca24e22eb1093776df904f87ca7ee415a620e96a183fb55e20bd1e78"} Jul 16 12:45:09.156469 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:09.156422 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xp6gr" event={"ID":"4d4d0198-ddd2-4fbe-a81b-a9ac12923570","Type":"ContainerStarted","Data":"a6cb4e8fe8c261aeb1202fb28a4b1674cc6964c25a6eb0092e8c65d1d7673c9f"} Jul 16 12:45:09.157938 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:09.157914 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-22ksl" event={"ID":"77529ff5-36ad-4040-8ec3-8a3bc3c01c94","Type":"ContainerStarted","Data":"b98bc078b3c0c2f0a91d5ae0a98b8a5ae0cc1ee2929e5a4a4f88e30b1cdeb01e"} Jul 16 12:45:09.158068 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:09.158011 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-22ksl" podUID="77529ff5-36ad-4040-8ec3-8a3bc3c01c94" containerName="registry-server" containerID="cri-o://b98bc078b3c0c2f0a91d5ae0a98b8a5ae0cc1ee2929e5a4a4f88e30b1cdeb01e" gracePeriod=2 Jul 16 12:45:09.179217 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:09.179178 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xp6gr" podStartSLOduration=1.432003422 podStartE2EDuration="3.179164972s" totalImagesPullingTime="1.74716155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:45:09.17672332 +0000 UTC m=+362.410631171" watchObservedRunningTime="2026-07-16 12:45:09.179164972 +0000 UTC m=+362.413072824" Jul 16 12:45:09.193237 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:09.193196 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-22ksl" podStartSLOduration=1.864671508 podStartE2EDuration="4.19318307s" totalImagesPullingTime="2.328511562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:45:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:45:09.192079339 +0000 UTC m=+362.425987191" watchObservedRunningTime="2026-07-16 12:45:09.19318307 +0000 UTC m=+362.427090920" Jul 16 12:45:09.401903 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:09.401876 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-22ksl" Jul 16 12:45:09.497986 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:09.497958 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l4fh\" (UniqueName: \"kubernetes.io/projected/77529ff5-36ad-4040-8ec3-8a3bc3c01c94-kube-api-access-8l4fh\") pod \"77529ff5-36ad-4040-8ec3-8a3bc3c01c94\" (UID: \"77529ff5-36ad-4040-8ec3-8a3bc3c01c94\") " Jul 16 12:45:09.500058 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:09.500014 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77529ff5-36ad-4040-8ec3-8a3bc3c01c94-kube-api-access-8l4fh" (OuterVolumeSpecName: "kube-api-access-8l4fh") pod "77529ff5-36ad-4040-8ec3-8a3bc3c01c94" (UID: "77529ff5-36ad-4040-8ec3-8a3bc3c01c94"). InnerVolumeSpecName "kube-api-access-8l4fh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:45:09.599002 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:09.598969 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l4fh\" (UniqueName: \"kubernetes.io/projected/77529ff5-36ad-4040-8ec3-8a3bc3c01c94-kube-api-access-8l4fh\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:45:10.165182 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:10.165143 2567 generic.go:356] "Generic (PLEG): container finished" podID="77529ff5-36ad-4040-8ec3-8a3bc3c01c94" containerID="b98bc078b3c0c2f0a91d5ae0a98b8a5ae0cc1ee2929e5a4a4f88e30b1cdeb01e" exitCode=0 Jul 16 12:45:10.165566 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:10.165200 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-22ksl" Jul 16 12:45:10.165566 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:10.165222 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-22ksl" event={"ID":"77529ff5-36ad-4040-8ec3-8a3bc3c01c94","Type":"ContainerDied","Data":"b98bc078b3c0c2f0a91d5ae0a98b8a5ae0cc1ee2929e5a4a4f88e30b1cdeb01e"} Jul 16 12:45:10.165566 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:10.165293 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-22ksl" event={"ID":"77529ff5-36ad-4040-8ec3-8a3bc3c01c94","Type":"ContainerDied","Data":"18c35b7eef6493ee7decf3e6c4e6fdb3391e9d41f165cebc013cc054b34e6703"} Jul 16 12:45:10.165566 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:10.165314 2567 scope.go:117] "RemoveContainer" containerID="b98bc078b3c0c2f0a91d5ae0a98b8a5ae0cc1ee2929e5a4a4f88e30b1cdeb01e" Jul 16 12:45:10.173617 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:10.173602 2567 scope.go:117] "RemoveContainer" containerID="b98bc078b3c0c2f0a91d5ae0a98b8a5ae0cc1ee2929e5a4a4f88e30b1cdeb01e" Jul 16 12:45:10.173876 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:45:10.173856 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b98bc078b3c0c2f0a91d5ae0a98b8a5ae0cc1ee2929e5a4a4f88e30b1cdeb01e\": container with ID starting with b98bc078b3c0c2f0a91d5ae0a98b8a5ae0cc1ee2929e5a4a4f88e30b1cdeb01e not found: ID does not exist" containerID="b98bc078b3c0c2f0a91d5ae0a98b8a5ae0cc1ee2929e5a4a4f88e30b1cdeb01e" Jul 16 12:45:10.173927 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:10.173884 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b98bc078b3c0c2f0a91d5ae0a98b8a5ae0cc1ee2929e5a4a4f88e30b1cdeb01e"} err="failed to get container status \"b98bc078b3c0c2f0a91d5ae0a98b8a5ae0cc1ee2929e5a4a4f88e30b1cdeb01e\": rpc error: code = NotFound desc = could not find container \"b98bc078b3c0c2f0a91d5ae0a98b8a5ae0cc1ee2929e5a4a4f88e30b1cdeb01e\": container with ID starting with b98bc078b3c0c2f0a91d5ae0a98b8a5ae0cc1ee2929e5a4a4f88e30b1cdeb01e not found: ID does not exist" Jul 16 12:45:10.181664 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:10.181643 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-22ksl"] Jul 16 12:45:10.185042 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:10.185012 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-22ksl"] Jul 16 12:45:11.639171 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:11.639140 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77529ff5-36ad-4040-8ec3-8a3bc3c01c94" path="/var/lib/kubelet/pods/77529ff5-36ad-4040-8ec3-8a3bc3c01c94/volumes" Jul 16 12:45:16.509115 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:16.509082 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-xp6gr" Jul 16 12:45:16.509563 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:16.509140 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-xp6gr" Jul 16 12:45:16.532097 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:16.532073 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-xp6gr" Jul 16 12:45:17.217755 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:17.217730 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-xp6gr" Jul 16 12:45:38.140316 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:38.140276 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lcp54"] Jul 16 12:45:38.140787 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:38.140586 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77529ff5-36ad-4040-8ec3-8a3bc3c01c94" containerName="registry-server" Jul 16 12:45:38.140787 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:38.140597 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="77529ff5-36ad-4040-8ec3-8a3bc3c01c94" containerName="registry-server" Jul 16 12:45:38.140787 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:38.140663 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="77529ff5-36ad-4040-8ec3-8a3bc3c01c94" containerName="registry-server" Jul 16 12:45:38.143550 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:38.143529 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lcp54" Jul 16 12:45:38.146136 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:38.146108 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-5ws65\"" Jul 16 12:45:38.153610 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:38.153586 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lcp54"] Jul 16 12:45:38.230303 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:38.230272 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87rzx\" (UniqueName: \"kubernetes.io/projected/6140e625-a1e4-4c56-a7b4-b9ae2cff409c-kube-api-access-87rzx\") pod \"authorino-operator-75cc8d5cb7-lcp54\" (UID: \"6140e625-a1e4-4c56-a7b4-b9ae2cff409c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lcp54" Jul 16 12:45:38.330867 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:38.330826 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87rzx\" (UniqueName: \"kubernetes.io/projected/6140e625-a1e4-4c56-a7b4-b9ae2cff409c-kube-api-access-87rzx\") pod \"authorino-operator-75cc8d5cb7-lcp54\" (UID: \"6140e625-a1e4-4c56-a7b4-b9ae2cff409c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lcp54" Jul 16 12:45:38.339619 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:38.339591 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87rzx\" (UniqueName: \"kubernetes.io/projected/6140e625-a1e4-4c56-a7b4-b9ae2cff409c-kube-api-access-87rzx\") pod \"authorino-operator-75cc8d5cb7-lcp54\" (UID: \"6140e625-a1e4-4c56-a7b4-b9ae2cff409c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lcp54" Jul 16 12:45:38.454488 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:38.454458 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lcp54" Jul 16 12:45:38.586488 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:38.586461 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lcp54"] Jul 16 12:45:38.589064 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:45:38.589012 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6140e625_a1e4_4c56_a7b4_b9ae2cff409c.slice/crio-d31f5c35ed1f4b4ffcc87765a6f60080f0d5251e03832c69dc4b5246bec109b5 WatchSource:0}: Error finding container d31f5c35ed1f4b4ffcc87765a6f60080f0d5251e03832c69dc4b5246bec109b5: Status 404 returned error can't find the container with id d31f5c35ed1f4b4ffcc87765a6f60080f0d5251e03832c69dc4b5246bec109b5 Jul 16 12:45:39.291795 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:39.291760 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lcp54" event={"ID":"6140e625-a1e4-4c56-a7b4-b9ae2cff409c","Type":"ContainerStarted","Data":"d31f5c35ed1f4b4ffcc87765a6f60080f0d5251e03832c69dc4b5246bec109b5"} Jul 16 12:45:41.303817 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:41.303778 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lcp54" event={"ID":"6140e625-a1e4-4c56-a7b4-b9ae2cff409c","Type":"ContainerStarted","Data":"66e73f6765855afd177e3515f6d05ebd22064ac6a423c7ad374fd6491f29f53f"} Jul 16 12:45:41.304236 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:41.303917 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lcp54" Jul 16 12:45:41.323914 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:41.323864 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lcp54" podStartSLOduration=1.04415683 podStartE2EDuration="3.323848195s" totalImagesPullingTime="2.279691365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:45:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:45:41.321363617 +0000 UTC m=+394.555271470" watchObservedRunningTime="2026-07-16 12:45:41.323848195 +0000 UTC m=+394.557756045" Jul 16 12:45:41.414302 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:41.414269 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5"] Jul 16 12:45:41.417608 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:41.417591 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:45:41.420315 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:41.420292 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-mb8f2\"" Jul 16 12:45:41.432118 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:41.432097 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5"] Jul 16 12:45:41.484544 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:41.484518 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr849\" (UniqueName: \"kubernetes.io/projected/8e5bb79b-99a1-481c-94ed-01da595e9020-kube-api-access-zr849\") pod \"limitador-operator-controller-manager-859d7b78bd-f7jz5\" (UID: \"8e5bb79b-99a1-481c-94ed-01da595e9020\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:45:41.585671 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:41.585589 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr849\" (UniqueName: \"kubernetes.io/projected/8e5bb79b-99a1-481c-94ed-01da595e9020-kube-api-access-zr849\") pod \"limitador-operator-controller-manager-859d7b78bd-f7jz5\" (UID: \"8e5bb79b-99a1-481c-94ed-01da595e9020\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:45:41.599471 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:41.599445 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr849\" (UniqueName: \"kubernetes.io/projected/8e5bb79b-99a1-481c-94ed-01da595e9020-kube-api-access-zr849\") pod \"limitador-operator-controller-manager-859d7b78bd-f7jz5\" (UID: \"8e5bb79b-99a1-481c-94ed-01da595e9020\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:45:41.727170 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:41.727127 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:45:41.858536 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:41.858474 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5"] Jul 16 12:45:41.860666 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:45:41.860636 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e5bb79b_99a1_481c_94ed_01da595e9020.slice/crio-ba1e0d6cee359a0d83673ac5e30c38021be53900c9506968b10ff525bd815da6 WatchSource:0}: Error finding container ba1e0d6cee359a0d83673ac5e30c38021be53900c9506968b10ff525bd815da6: Status 404 returned error can't find the container with id ba1e0d6cee359a0d83673ac5e30c38021be53900c9506968b10ff525bd815da6 Jul 16 12:45:42.308990 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:42.308958 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" event={"ID":"8e5bb79b-99a1-481c-94ed-01da595e9020","Type":"ContainerStarted","Data":"ba1e0d6cee359a0d83673ac5e30c38021be53900c9506968b10ff525bd815da6"} Jul 16 12:45:44.320193 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:44.320152 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" event={"ID":"8e5bb79b-99a1-481c-94ed-01da595e9020","Type":"ContainerStarted","Data":"778d1e7f0b8a0f95d3b3b7870ff86063f78ac4668b5f9b6ee9e0955dd4052d5c"} Jul 16 12:45:44.320720 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:44.320256 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:45:44.339693 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:44.339645 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" podStartSLOduration=1.7659629620000001 podStartE2EDuration="3.339632133s" totalImagesPullingTime="1.573669171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:45:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:45:44.338080769 +0000 UTC m=+397.571988621" watchObservedRunningTime="2026-07-16 12:45:44.339632133 +0000 UTC m=+397.573539984" Jul 16 12:45:46.238932 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:46.238900 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l"] Jul 16 12:45:46.242492 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:46.242470 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:45:46.247753 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:46.247720 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nxk26\"" Jul 16 12:45:46.256705 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:46.256683 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l"] Jul 16 12:45:46.323344 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:46.323311 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/203ca8aa-f401-47b9-b5c2-ac9ee2837b7e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pth9l\" (UID: \"203ca8aa-f401-47b9-b5c2-ac9ee2837b7e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:45:46.323496 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:46.323376 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v8hz\" (UniqueName: \"kubernetes.io/projected/203ca8aa-f401-47b9-b5c2-ac9ee2837b7e-kube-api-access-2v8hz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pth9l\" (UID: \"203ca8aa-f401-47b9-b5c2-ac9ee2837b7e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:45:46.423931 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:46.423896 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v8hz\" (UniqueName: \"kubernetes.io/projected/203ca8aa-f401-47b9-b5c2-ac9ee2837b7e-kube-api-access-2v8hz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pth9l\" (UID: \"203ca8aa-f401-47b9-b5c2-ac9ee2837b7e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:45:46.424122 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:46.423952 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/203ca8aa-f401-47b9-b5c2-ac9ee2837b7e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pth9l\" (UID: \"203ca8aa-f401-47b9-b5c2-ac9ee2837b7e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:45:46.424346 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:46.424327 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/203ca8aa-f401-47b9-b5c2-ac9ee2837b7e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pth9l\" (UID: \"203ca8aa-f401-47b9-b5c2-ac9ee2837b7e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:45:46.435954 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:46.435928 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v8hz\" (UniqueName: \"kubernetes.io/projected/203ca8aa-f401-47b9-b5c2-ac9ee2837b7e-kube-api-access-2v8hz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pth9l\" (UID: \"203ca8aa-f401-47b9-b5c2-ac9ee2837b7e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:45:46.552708 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:46.552639 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:45:46.685170 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:46.684999 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l"] Jul 16 12:45:46.688041 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:45:46.688012 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod203ca8aa_f401_47b9_b5c2_ac9ee2837b7e.slice/crio-b82a8e2c2422c62d0c521528ad9b5a52dccb498d46d6288dd42f74c81e0bd57f WatchSource:0}: Error finding container b82a8e2c2422c62d0c521528ad9b5a52dccb498d46d6288dd42f74c81e0bd57f: Status 404 returned error can't find the container with id b82a8e2c2422c62d0c521528ad9b5a52dccb498d46d6288dd42f74c81e0bd57f Jul 16 12:45:47.336623 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:47.336586 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" event={"ID":"203ca8aa-f401-47b9-b5c2-ac9ee2837b7e","Type":"ContainerStarted","Data":"b82a8e2c2422c62d0c521528ad9b5a52dccb498d46d6288dd42f74c81e0bd57f"} Jul 16 12:45:49.352276 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:49.352232 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gzhrc"] Jul 16 12:45:49.355865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:49.355836 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gzhrc" Jul 16 12:45:49.358698 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:49.358672 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-dlfx6\"" Jul 16 12:45:49.358854 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:49.358672 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 16 12:45:49.364701 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:49.364674 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gzhrc"] Jul 16 12:45:49.450521 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:49.450474 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pksmf\" (UniqueName: \"kubernetes.io/projected/6326a8bf-e22e-420a-9981-811c3b238cb7-kube-api-access-pksmf\") pod \"dns-operator-controller-manager-864dd78c4c-gzhrc\" (UID: \"6326a8bf-e22e-420a-9981-811c3b238cb7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gzhrc" Jul 16 12:45:49.551062 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:49.550997 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pksmf\" (UniqueName: \"kubernetes.io/projected/6326a8bf-e22e-420a-9981-811c3b238cb7-kube-api-access-pksmf\") pod \"dns-operator-controller-manager-864dd78c4c-gzhrc\" (UID: \"6326a8bf-e22e-420a-9981-811c3b238cb7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gzhrc" Jul 16 12:45:49.560980 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:49.560917 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pksmf\" (UniqueName: \"kubernetes.io/projected/6326a8bf-e22e-420a-9981-811c3b238cb7-kube-api-access-pksmf\") pod \"dns-operator-controller-manager-864dd78c4c-gzhrc\" (UID: \"6326a8bf-e22e-420a-9981-811c3b238cb7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gzhrc" Jul 16 12:45:49.670713 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:49.670627 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gzhrc" Jul 16 12:45:49.838678 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:49.838646 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gzhrc"] Jul 16 12:45:49.842641 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:45:49.842610 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6326a8bf_e22e_420a_9981_811c3b238cb7.slice/crio-84580ab2549b24fe1552ef9a1bd6e258b5330e8319c95b715000cfd7e47892c9 WatchSource:0}: Error finding container 84580ab2549b24fe1552ef9a1bd6e258b5330e8319c95b715000cfd7e47892c9: Status 404 returned error can't find the container with id 84580ab2549b24fe1552ef9a1bd6e258b5330e8319c95b715000cfd7e47892c9 Jul 16 12:45:50.353292 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:50.353255 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gzhrc" event={"ID":"6326a8bf-e22e-420a-9981-811c3b238cb7","Type":"ContainerStarted","Data":"84580ab2549b24fe1552ef9a1bd6e258b5330e8319c95b715000cfd7e47892c9"} Jul 16 12:45:52.311564 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:52.311534 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lcp54" Jul 16 12:45:53.374979 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:53.374939 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" event={"ID":"203ca8aa-f401-47b9-b5c2-ac9ee2837b7e","Type":"ContainerStarted","Data":"49de925498edcc35a38651931d3fadd07d5fab32bc5b914d5e8d967f218699fc"} Jul 16 12:45:53.375379 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:53.375110 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:45:53.376563 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:53.376539 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gzhrc" event={"ID":"6326a8bf-e22e-420a-9981-811c3b238cb7","Type":"ContainerStarted","Data":"33f8d758abcfdc9ba801b4ca504ed882a0b63664c5dd2a955599c1294b883a42"} Jul 16 12:45:53.376672 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:53.376597 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gzhrc" Jul 16 12:45:53.398486 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:53.398437 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" podStartSLOduration=1.319233132 podStartE2EDuration="7.398423866s" totalImagesPullingTime="6.079190734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:45:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:45:53.396388863 +0000 UTC m=+406.630296717" watchObservedRunningTime="2026-07-16 12:45:53.398423866 +0000 UTC m=+406.632331792" Jul 16 12:45:53.414346 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:53.414308 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gzhrc" podStartSLOduration=1.494878089 podStartE2EDuration="4.414295065s" totalImagesPullingTime="2.919416976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:45:53.413583838 +0000 UTC m=+406.647491689" watchObservedRunningTime="2026-07-16 12:45:53.414295065 +0000 UTC m=+406.648202916" Jul 16 12:45:55.327762 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:55.327726 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:45:55.725205 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:55.725172 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll"] Jul 16 12:45:55.728610 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:55.728587 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll" Jul 16 12:45:55.731037 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:55.731008 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 16 12:45:55.731339 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:55.731321 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 16 12:45:55.731431 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:55.731325 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-w5vrd\"" Jul 16 12:45:55.738206 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:55.738186 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll"] Jul 16 12:45:55.806002 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:55.805968 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/07e4e793-4349-48b9-b10b-9d16821b91ec-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-n8zll\" (UID: \"07e4e793-4349-48b9-b10b-9d16821b91ec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll" Jul 16 12:45:55.806166 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:55.806079 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/07e4e793-4349-48b9-b10b-9d16821b91ec-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-n8zll\" (UID: \"07e4e793-4349-48b9-b10b-9d16821b91ec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll" Jul 16 12:45:55.806166 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:55.806126 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdpcb\" (UniqueName: \"kubernetes.io/projected/07e4e793-4349-48b9-b10b-9d16821b91ec-kube-api-access-tdpcb\") pod \"kuadrant-console-plugin-66b5774985-n8zll\" (UID: \"07e4e793-4349-48b9-b10b-9d16821b91ec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll" Jul 16 12:45:55.907236 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:55.907205 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/07e4e793-4349-48b9-b10b-9d16821b91ec-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-n8zll\" (UID: \"07e4e793-4349-48b9-b10b-9d16821b91ec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll" Jul 16 12:45:55.907405 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:55.907244 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdpcb\" (UniqueName: \"kubernetes.io/projected/07e4e793-4349-48b9-b10b-9d16821b91ec-kube-api-access-tdpcb\") pod \"kuadrant-console-plugin-66b5774985-n8zll\" (UID: \"07e4e793-4349-48b9-b10b-9d16821b91ec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll" Jul 16 12:45:55.907405 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:55.907277 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/07e4e793-4349-48b9-b10b-9d16821b91ec-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-n8zll\" (UID: \"07e4e793-4349-48b9-b10b-9d16821b91ec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll" Jul 16 12:45:55.907865 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:55.907846 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/07e4e793-4349-48b9-b10b-9d16821b91ec-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-n8zll\" (UID: \"07e4e793-4349-48b9-b10b-9d16821b91ec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll" Jul 16 12:45:55.909585 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:55.909566 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/07e4e793-4349-48b9-b10b-9d16821b91ec-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-n8zll\" (UID: \"07e4e793-4349-48b9-b10b-9d16821b91ec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll" Jul 16 12:45:55.918251 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:55.918225 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdpcb\" (UniqueName: \"kubernetes.io/projected/07e4e793-4349-48b9-b10b-9d16821b91ec-kube-api-access-tdpcb\") pod \"kuadrant-console-plugin-66b5774985-n8zll\" (UID: \"07e4e793-4349-48b9-b10b-9d16821b91ec\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll" Jul 16 12:45:56.038715 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:56.038633 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll" Jul 16 12:45:56.169500 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:56.169341 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll"] Jul 16 12:45:56.171495 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:45:56.171469 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07e4e793_4349_48b9_b10b_9d16821b91ec.slice/crio-6c2051d7b20c463f75140d49ea2db115c3cf03be812e1a7596c88633bacf2959 WatchSource:0}: Error finding container 6c2051d7b20c463f75140d49ea2db115c3cf03be812e1a7596c88633bacf2959: Status 404 returned error can't find the container with id 6c2051d7b20c463f75140d49ea2db115c3cf03be812e1a7596c88633bacf2959 Jul 16 12:45:56.392922 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:45:56.392834 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll" event={"ID":"07e4e793-4349-48b9-b10b-9d16821b91ec","Type":"ContainerStarted","Data":"6c2051d7b20c463f75140d49ea2db115c3cf03be812e1a7596c88633bacf2959"} Jul 16 12:46:04.383800 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:04.383767 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:46:04.384242 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:04.383890 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gzhrc" Jul 16 12:46:06.425237 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.425201 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l"] Jul 16 12:46:06.425683 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.425427 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" podUID="203ca8aa-f401-47b9-b5c2-ac9ee2837b7e" containerName="manager" containerID="cri-o://49de925498edcc35a38651931d3fadd07d5fab32bc5b914d5e8d967f218699fc" gracePeriod=2 Jul 16 12:46:06.457816 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:06.457769 2567 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"203ca8aa-f401-47b9-b5c2-ac9ee2837b7e\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-pth9l\": pods \"kuadrant-operator-controller-manager-8684c9ff49-pth9l\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:46:06.458664 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.458522 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l"] Jul 16 12:46:06.470030 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.469997 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5"] Jul 16 12:46:06.470331 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.470259 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" podUID="8e5bb79b-99a1-481c-94ed-01da595e9020" containerName="manager" containerID="cri-o://778d1e7f0b8a0f95d3b3b7870ff86063f78ac4668b5f9b6ee9e0955dd4052d5c" gracePeriod=2 Jul 16 12:46:06.473289 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:06.473236 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pth9l\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="203ca8aa-f401-47b9-b5c2-ac9ee2837b7e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:46:06.476313 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.476262 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5"] Jul 16 12:46:06.482037 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:06.482006 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f7jz5\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="8e5bb79b-99a1-481c-94ed-01da595e9020" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:46:06.497222 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.497194 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx"] Jul 16 12:46:06.497783 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.497676 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e5bb79b-99a1-481c-94ed-01da595e9020" containerName="manager" Jul 16 12:46:06.497783 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.497698 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e5bb79b-99a1-481c-94ed-01da595e9020" containerName="manager" Jul 16 12:46:06.497783 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.497720 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="203ca8aa-f401-47b9-b5c2-ac9ee2837b7e" containerName="manager" Jul 16 12:46:06.497783 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.497728 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="203ca8aa-f401-47b9-b5c2-ac9ee2837b7e" containerName="manager" Jul 16 12:46:06.498163 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.497837 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e5bb79b-99a1-481c-94ed-01da595e9020" containerName="manager" Jul 16 12:46:06.498163 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.497856 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="203ca8aa-f401-47b9-b5c2-ac9ee2837b7e" containerName="manager" Jul 16 12:46:06.501003 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.500985 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" Jul 16 12:46:06.504651 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:06.504613 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f7jz5\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="8e5bb79b-99a1-481c-94ed-01da595e9020" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:46:06.528539 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.528490 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vh8c"] Jul 16 12:46:06.537077 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.537022 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vh8c" Jul 16 12:46:06.541462 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.541437 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx"] Jul 16 12:46:06.573673 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.573644 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vh8c"] Jul 16 12:46:06.589106 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:06.589071 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pth9l\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="203ca8aa-f401-47b9-b5c2-ac9ee2837b7e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:46:06.606191 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.606165 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kgcdx\" (UID: \"3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" Jul 16 12:46:06.606310 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.606246 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs944\" (UniqueName: \"kubernetes.io/projected/4d997cae-661d-4489-88fe-d4db001fd5ce-kube-api-access-hs944\") pod \"limitador-operator-controller-manager-859d7b78bd-4vh8c\" (UID: \"4d997cae-661d-4489-88fe-d4db001fd5ce\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vh8c" Jul 16 12:46:06.606310 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.606279 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq6vt\" (UniqueName: \"kubernetes.io/projected/3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d-kube-api-access-jq6vt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kgcdx\" (UID: \"3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" Jul 16 12:46:06.614008 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:06.613967 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pth9l\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="203ca8aa-f401-47b9-b5c2-ac9ee2837b7e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:46:06.616253 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:06.616227 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f7jz5\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="8e5bb79b-99a1-481c-94ed-01da595e9020" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:46:06.707389 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.707340 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs944\" (UniqueName: \"kubernetes.io/projected/4d997cae-661d-4489-88fe-d4db001fd5ce-kube-api-access-hs944\") pod \"limitador-operator-controller-manager-859d7b78bd-4vh8c\" (UID: \"4d997cae-661d-4489-88fe-d4db001fd5ce\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vh8c" Jul 16 12:46:06.707569 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.707397 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq6vt\" (UniqueName: \"kubernetes.io/projected/3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d-kube-api-access-jq6vt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kgcdx\" (UID: \"3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" Jul 16 12:46:06.708127 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.707671 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kgcdx\" (UID: \"3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" Jul 16 12:46:06.708322 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.708033 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kgcdx\" (UID: \"3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" Jul 16 12:46:06.716912 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.716888 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq6vt\" (UniqueName: \"kubernetes.io/projected/3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d-kube-api-access-jq6vt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kgcdx\" (UID: \"3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" Jul 16 12:46:06.717280 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.717264 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs944\" (UniqueName: \"kubernetes.io/projected/4d997cae-661d-4489-88fe-d4db001fd5ce-kube-api-access-hs944\") pod \"limitador-operator-controller-manager-859d7b78bd-4vh8c\" (UID: \"4d997cae-661d-4489-88fe-d4db001fd5ce\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vh8c" Jul 16 12:46:06.862072 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.862026 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" Jul 16 12:46:06.869188 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:06.869160 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vh8c" Jul 16 12:46:07.638758 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:07.638680 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f7jz5\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="8e5bb79b-99a1-481c-94ed-01da595e9020" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:46:07.640796 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:07.640761 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pth9l\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="203ca8aa-f401-47b9-b5c2-ac9ee2837b7e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:46:08.793853 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.793754 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-ccc56667d-725m6"] Jul 16 12:46:08.850756 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.850717 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ccc56667d-725m6"] Jul 16 12:46:08.850918 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.850848 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:08.853684 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.853660 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 12:46:08.855060 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.855015 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 12:46:08.855060 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.855034 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 12:46:08.855241 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.855102 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 12:46:08.855241 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.855021 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kv8xc\"" Jul 16 12:46:08.855331 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.855276 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 12:46:08.855331 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.855295 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 12:46:08.855425 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.855360 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 12:46:08.859391 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.859331 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 12:46:08.931447 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.931406 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7254a9a0-5304-415c-86a5-f28c720ed7a4-console-oauth-config\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:08.931635 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.931556 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7254a9a0-5304-415c-86a5-f28c720ed7a4-oauth-serving-cert\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:08.931721 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.931631 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7254a9a0-5304-415c-86a5-f28c720ed7a4-trusted-ca-bundle\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:08.931721 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.931687 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hfg5\" (UniqueName: \"kubernetes.io/projected/7254a9a0-5304-415c-86a5-f28c720ed7a4-kube-api-access-4hfg5\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:08.931828 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.931738 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7254a9a0-5304-415c-86a5-f28c720ed7a4-console-serving-cert\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:08.931828 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.931764 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7254a9a0-5304-415c-86a5-f28c720ed7a4-console-config\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:08.931992 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:08.931836 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7254a9a0-5304-415c-86a5-f28c720ed7a4-service-ca\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:09.033025 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:09.032988 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7254a9a0-5304-415c-86a5-f28c720ed7a4-console-serving-cert\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:09.033025 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:09.033032 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7254a9a0-5304-415c-86a5-f28c720ed7a4-console-config\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:09.033276 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:09.033080 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7254a9a0-5304-415c-86a5-f28c720ed7a4-service-ca\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:09.033276 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:09.033121 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7254a9a0-5304-415c-86a5-f28c720ed7a4-console-oauth-config\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:09.033276 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:09.033178 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7254a9a0-5304-415c-86a5-f28c720ed7a4-oauth-serving-cert\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:09.033276 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:09.033226 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7254a9a0-5304-415c-86a5-f28c720ed7a4-trusted-ca-bundle\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:09.033276 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:09.033263 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hfg5\" (UniqueName: \"kubernetes.io/projected/7254a9a0-5304-415c-86a5-f28c720ed7a4-kube-api-access-4hfg5\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:09.033837 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:09.033789 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7254a9a0-5304-415c-86a5-f28c720ed7a4-console-config\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:09.034085 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:09.034031 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7254a9a0-5304-415c-86a5-f28c720ed7a4-oauth-serving-cert\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:09.034210 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:09.034187 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7254a9a0-5304-415c-86a5-f28c720ed7a4-trusted-ca-bundle\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:09.034608 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:09.034589 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7254a9a0-5304-415c-86a5-f28c720ed7a4-service-ca\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:09.035843 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:09.035820 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7254a9a0-5304-415c-86a5-f28c720ed7a4-console-oauth-config\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:09.035958 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:09.035939 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7254a9a0-5304-415c-86a5-f28c720ed7a4-console-serving-cert\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:09.047246 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:09.047175 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hfg5\" (UniqueName: \"kubernetes.io/projected/7254a9a0-5304-415c-86a5-f28c720ed7a4-kube-api-access-4hfg5\") pod \"console-ccc56667d-725m6\" (UID: \"7254a9a0-5304-415c-86a5-f28c720ed7a4\") " pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:09.163058 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:09.162996 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:18.548368 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:18.548209 2567 generic.go:356] "Generic (PLEG): container finished" podID="203ca8aa-f401-47b9-b5c2-ac9ee2837b7e" containerID="49de925498edcc35a38651931d3fadd07d5fab32bc5b914d5e8d967f218699fc" exitCode=0 Jul 16 12:46:18.551463 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:18.550470 2567 generic.go:356] "Generic (PLEG): container finished" podID="8e5bb79b-99a1-481c-94ed-01da595e9020" containerID="778d1e7f0b8a0f95d3b3b7870ff86063f78ac4668b5f9b6ee9e0955dd4052d5c" exitCode=0 Jul 16 12:46:18.613147 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:18.612804 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vh8c"] Jul 16 12:46:18.613473 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:46:18.613387 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d997cae_661d_4489_88fe_d4db001fd5ce.slice/crio-f7ad197691df1aff2cd79038eb9c031e63b908eddfbaac532e3cc42915036282 WatchSource:0}: Error finding container f7ad197691df1aff2cd79038eb9c031e63b908eddfbaac532e3cc42915036282: Status 404 returned error can't find the container with id f7ad197691df1aff2cd79038eb9c031e63b908eddfbaac532e3cc42915036282 Jul 16 12:46:18.622973 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:18.622950 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:46:18.625042 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:18.625020 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:46:18.625366 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:18.625336 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f7jz5\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="8e5bb79b-99a1-481c-94ed-01da595e9020" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:46:18.627464 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:18.627436 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f7jz5\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="8e5bb79b-99a1-481c-94ed-01da595e9020" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:46:18.629377 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:18.629333 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pth9l\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="203ca8aa-f401-47b9-b5c2-ac9ee2837b7e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:46:18.723504 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:18.723482 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2v8hz\" (UniqueName: \"kubernetes.io/projected/203ca8aa-f401-47b9-b5c2-ac9ee2837b7e-kube-api-access-2v8hz\") pod \"203ca8aa-f401-47b9-b5c2-ac9ee2837b7e\" (UID: \"203ca8aa-f401-47b9-b5c2-ac9ee2837b7e\") " Jul 16 12:46:18.723602 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:18.723580 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/203ca8aa-f401-47b9-b5c2-ac9ee2837b7e-extensions-socket-volume\") pod \"203ca8aa-f401-47b9-b5c2-ac9ee2837b7e\" (UID: \"203ca8aa-f401-47b9-b5c2-ac9ee2837b7e\") " Jul 16 12:46:18.723644 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:18.723615 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zr849\" (UniqueName: \"kubernetes.io/projected/8e5bb79b-99a1-481c-94ed-01da595e9020-kube-api-access-zr849\") pod \"8e5bb79b-99a1-481c-94ed-01da595e9020\" (UID: \"8e5bb79b-99a1-481c-94ed-01da595e9020\") " Jul 16 12:46:18.724177 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:18.724147 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/203ca8aa-f401-47b9-b5c2-ac9ee2837b7e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "203ca8aa-f401-47b9-b5c2-ac9ee2837b7e" (UID: "203ca8aa-f401-47b9-b5c2-ac9ee2837b7e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:46:18.725415 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:18.725387 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/203ca8aa-f401-47b9-b5c2-ac9ee2837b7e-kube-api-access-2v8hz" (OuterVolumeSpecName: "kube-api-access-2v8hz") pod "203ca8aa-f401-47b9-b5c2-ac9ee2837b7e" (UID: "203ca8aa-f401-47b9-b5c2-ac9ee2837b7e"). InnerVolumeSpecName "kube-api-access-2v8hz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:46:18.725577 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:18.725557 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e5bb79b-99a1-481c-94ed-01da595e9020-kube-api-access-zr849" (OuterVolumeSpecName: "kube-api-access-zr849") pod "8e5bb79b-99a1-481c-94ed-01da595e9020" (UID: "8e5bb79b-99a1-481c-94ed-01da595e9020"). InnerVolumeSpecName "kube-api-access-zr849". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:46:18.825171 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:18.825147 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/203ca8aa-f401-47b9-b5c2-ac9ee2837b7e-extensions-socket-volume\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:46:18.825171 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:18.825170 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zr849\" (UniqueName: \"kubernetes.io/projected/8e5bb79b-99a1-481c-94ed-01da595e9020-kube-api-access-zr849\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:46:18.825338 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:18.825180 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2v8hz\" (UniqueName: \"kubernetes.io/projected/203ca8aa-f401-47b9-b5c2-ac9ee2837b7e-kube-api-access-2v8hz\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:46:18.851171 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:18.851104 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ccc56667d-725m6"] Jul 16 12:46:18.854193 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:18.854168 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx"] Jul 16 12:46:18.854193 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:46:18.854180 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7254a9a0_5304_415c_86a5_f28c720ed7a4.slice/crio-01213ea104575c350f379643d32f5d2357b4c015aeaac9f197de49a6b1056f05 WatchSource:0}: Error finding container 01213ea104575c350f379643d32f5d2357b4c015aeaac9f197de49a6b1056f05: Status 404 returned error can't find the container with id 01213ea104575c350f379643d32f5d2357b4c015aeaac9f197de49a6b1056f05 Jul 16 12:46:18.857694 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:46:18.857668 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bc42ec1_522b_4bc3_8ab9_ffade9c6a75d.slice/crio-ab1e940e47c382dd4ec1ea2e3a80a2adfa431b2541086dddeef1a55259a6cb45 WatchSource:0}: Error finding container ab1e940e47c382dd4ec1ea2e3a80a2adfa431b2541086dddeef1a55259a6cb45: Status 404 returned error can't find the container with id ab1e940e47c382dd4ec1ea2e3a80a2adfa431b2541086dddeef1a55259a6cb45 Jul 16 12:46:19.557210 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.557172 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ccc56667d-725m6" event={"ID":"7254a9a0-5304-415c-86a5-f28c720ed7a4","Type":"ContainerStarted","Data":"d307ff2e16ec8dd8578fe3183cd4ff5f7e0e84c1cdb20eeeb0c00b5fec9abce1"} Jul 16 12:46:19.557607 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.557231 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ccc56667d-725m6" event={"ID":"7254a9a0-5304-415c-86a5-f28c720ed7a4","Type":"ContainerStarted","Data":"01213ea104575c350f379643d32f5d2357b4c015aeaac9f197de49a6b1056f05"} Jul 16 12:46:19.558679 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.558663 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:46:19.558679 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.558677 2567 scope.go:117] "RemoveContainer" containerID="778d1e7f0b8a0f95d3b3b7870ff86063f78ac4668b5f9b6ee9e0955dd4052d5c" Jul 16 12:46:19.559515 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:19.559485 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pth9l\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="203ca8aa-f401-47b9-b5c2-ac9ee2837b7e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:46:19.560384 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.560362 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll" event={"ID":"07e4e793-4349-48b9-b10b-9d16821b91ec","Type":"ContainerStarted","Data":"84d9790a31bc52ade294ec09a51a67ec99672a236f409b457c124f51deb75673"} Jul 16 12:46:19.562171 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.562141 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" event={"ID":"3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d","Type":"ContainerStarted","Data":"c4094e0d5dae35c8b7813429f41ac4d6abc1b3460f30fffc0e7691759bdff9d9"} Jul 16 12:46:19.562293 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.562200 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" event={"ID":"3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d","Type":"ContainerStarted","Data":"ab1e940e47c382dd4ec1ea2e3a80a2adfa431b2541086dddeef1a55259a6cb45"} Jul 16 12:46:19.562293 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.562231 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" Jul 16 12:46:19.564006 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.563981 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vh8c" event={"ID":"4d997cae-661d-4489-88fe-d4db001fd5ce","Type":"ContainerStarted","Data":"e3c27db94ef92dba27ba21c5185c73c249220468017784e697cb5b9e4189436e"} Jul 16 12:46:19.564132 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.564033 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vh8c" event={"ID":"4d997cae-661d-4489-88fe-d4db001fd5ce","Type":"ContainerStarted","Data":"f7ad197691df1aff2cd79038eb9c031e63b908eddfbaac532e3cc42915036282"} Jul 16 12:46:19.564132 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.564104 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vh8c" Jul 16 12:46:19.565358 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.565339 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:46:19.568565 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.568550 2567 scope.go:117] "RemoveContainer" containerID="49de925498edcc35a38651931d3fadd07d5fab32bc5b914d5e8d967f218699fc" Jul 16 12:46:19.585422 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:19.585385 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f7jz5\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="8e5bb79b-99a1-481c-94ed-01da595e9020" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:46:19.587683 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:19.587652 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-pth9l\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="203ca8aa-f401-47b9-b5c2-ac9ee2837b7e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pth9l" Jul 16 12:46:19.608222 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.608180 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-ccc56667d-725m6" podStartSLOduration=11.608166922 podStartE2EDuration="11.608166922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:46:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:46:19.583287458 +0000 UTC m=+432.817195311" watchObservedRunningTime="2026-07-16 12:46:19.608166922 +0000 UTC m=+432.842074819" Jul 16 12:46:19.608483 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.608456 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-n8zll" podStartSLOduration=2.244083235 podStartE2EDuration="24.608449416s" totalImagesPullingTime="22.364366181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:45:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:46:19.60781892 +0000 UTC m=+432.841726789" watchObservedRunningTime="2026-07-16 12:46:19.608449416 +0000 UTC m=+432.842357267" Jul 16 12:46:19.610241 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:19.610216 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f7jz5\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="8e5bb79b-99a1-481c-94ed-01da595e9020" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f7jz5" Jul 16 12:46:19.634683 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.634642 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" podStartSLOduration=13.634628177 podStartE2EDuration="13.634628177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:46:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:46:19.633534893 +0000 UTC m=+432.867442746" watchObservedRunningTime="2026-07-16 12:46:19.634628177 +0000 UTC m=+432.868536028" Jul 16 12:46:19.640762 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.640736 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="203ca8aa-f401-47b9-b5c2-ac9ee2837b7e" path="/var/lib/kubelet/pods/203ca8aa-f401-47b9-b5c2-ac9ee2837b7e/volumes" Jul 16 12:46:19.641132 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.641116 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e5bb79b-99a1-481c-94ed-01da595e9020" path="/var/lib/kubelet/pods/8e5bb79b-99a1-481c-94ed-01da595e9020/volumes" Jul 16 12:46:19.650498 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:19.650455 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vh8c" podStartSLOduration=13.650442338 podStartE2EDuration="13.650442338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:46:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:46:19.649471048 +0000 UTC m=+432.883378901" watchObservedRunningTime="2026-07-16 12:46:19.650442338 +0000 UTC m=+432.884350190" Jul 16 12:46:22.095493 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.095459 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx"] Jul 16 12:46:22.095888 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.095631 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" podUID="3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d" containerName="manager" containerID="cri-o://c4094e0d5dae35c8b7813429f41ac4d6abc1b3460f30fffc0e7691759bdff9d9" gracePeriod=10 Jul 16 12:46:22.310224 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.310194 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259"] Jul 16 12:46:22.313645 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.313620 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" Jul 16 12:46:22.331135 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.331065 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259"] Jul 16 12:46:22.339058 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.339025 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" Jul 16 12:46:22.457881 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.457858 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jq6vt\" (UniqueName: \"kubernetes.io/projected/3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d-kube-api-access-jq6vt\") pod \"3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d\" (UID: \"3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d\") " Jul 16 12:46:22.458064 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.457926 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d-extensions-socket-volume\") pod \"3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d\" (UID: \"3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d\") " Jul 16 12:46:22.458119 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.458041 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/984f0cde-2733-42c7-b417-a2556ae98084-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ml259\" (UID: \"984f0cde-2733-42c7-b417-a2556ae98084\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" Jul 16 12:46:22.458224 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.458201 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jsh8\" (UniqueName: \"kubernetes.io/projected/984f0cde-2733-42c7-b417-a2556ae98084-kube-api-access-9jsh8\") pod \"kuadrant-operator-controller-manager-86878c769f-ml259\" (UID: \"984f0cde-2733-42c7-b417-a2556ae98084\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" Jul 16 12:46:22.458355 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.458336 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d" (UID: "3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:46:22.459897 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.459874 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d-kube-api-access-jq6vt" (OuterVolumeSpecName: "kube-api-access-jq6vt") pod "3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d" (UID: "3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d"). InnerVolumeSpecName "kube-api-access-jq6vt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:46:22.558900 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.558865 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/984f0cde-2733-42c7-b417-a2556ae98084-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ml259\" (UID: \"984f0cde-2733-42c7-b417-a2556ae98084\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" Jul 16 12:46:22.559035 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.558940 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jsh8\" (UniqueName: \"kubernetes.io/projected/984f0cde-2733-42c7-b417-a2556ae98084-kube-api-access-9jsh8\") pod \"kuadrant-operator-controller-manager-86878c769f-ml259\" (UID: \"984f0cde-2733-42c7-b417-a2556ae98084\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" Jul 16 12:46:22.559035 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.558974 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jq6vt\" (UniqueName: \"kubernetes.io/projected/3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d-kube-api-access-jq6vt\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:46:22.559035 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.558983 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d-extensions-socket-volume\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:46:22.559327 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.559305 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/984f0cde-2733-42c7-b417-a2556ae98084-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ml259\" (UID: \"984f0cde-2733-42c7-b417-a2556ae98084\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" Jul 16 12:46:22.568313 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.568290 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jsh8\" (UniqueName: \"kubernetes.io/projected/984f0cde-2733-42c7-b417-a2556ae98084-kube-api-access-9jsh8\") pod \"kuadrant-operator-controller-manager-86878c769f-ml259\" (UID: \"984f0cde-2733-42c7-b417-a2556ae98084\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" Jul 16 12:46:22.582798 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.582777 2567 generic.go:356] "Generic (PLEG): container finished" podID="3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d" containerID="c4094e0d5dae35c8b7813429f41ac4d6abc1b3460f30fffc0e7691759bdff9d9" exitCode=0 Jul 16 12:46:22.582899 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.582807 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" event={"ID":"3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d","Type":"ContainerDied","Data":"c4094e0d5dae35c8b7813429f41ac4d6abc1b3460f30fffc0e7691759bdff9d9"} Jul 16 12:46:22.582899 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.582863 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" event={"ID":"3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d","Type":"ContainerDied","Data":"ab1e940e47c382dd4ec1ea2e3a80a2adfa431b2541086dddeef1a55259a6cb45"} Jul 16 12:46:22.582899 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.582887 2567 scope.go:117] "RemoveContainer" containerID="c4094e0d5dae35c8b7813429f41ac4d6abc1b3460f30fffc0e7691759bdff9d9" Jul 16 12:46:22.582899 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.582821 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx" Jul 16 12:46:22.591162 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.591105 2567 scope.go:117] "RemoveContainer" containerID="c4094e0d5dae35c8b7813429f41ac4d6abc1b3460f30fffc0e7691759bdff9d9" Jul 16 12:46:22.591408 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:46:22.591387 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4094e0d5dae35c8b7813429f41ac4d6abc1b3460f30fffc0e7691759bdff9d9\": container with ID starting with c4094e0d5dae35c8b7813429f41ac4d6abc1b3460f30fffc0e7691759bdff9d9 not found: ID does not exist" containerID="c4094e0d5dae35c8b7813429f41ac4d6abc1b3460f30fffc0e7691759bdff9d9" Jul 16 12:46:22.591519 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.591418 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4094e0d5dae35c8b7813429f41ac4d6abc1b3460f30fffc0e7691759bdff9d9"} err="failed to get container status \"c4094e0d5dae35c8b7813429f41ac4d6abc1b3460f30fffc0e7691759bdff9d9\": rpc error: code = NotFound desc = could not find container \"c4094e0d5dae35c8b7813429f41ac4d6abc1b3460f30fffc0e7691759bdff9d9\": container with ID starting with c4094e0d5dae35c8b7813429f41ac4d6abc1b3460f30fffc0e7691759bdff9d9 not found: ID does not exist" Jul 16 12:46:22.610521 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.610498 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx"] Jul 16 12:46:22.614873 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.614852 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kgcdx"] Jul 16 12:46:22.635466 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.635447 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" Jul 16 12:46:22.773777 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:22.773753 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259"] Jul 16 12:46:22.775851 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:46:22.775823 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod984f0cde_2733_42c7_b417_a2556ae98084.slice/crio-06f5350f2bfaef7c34a47e047a2270fd58d005fc73ce53ceda4c4cdd37fb81c6 WatchSource:0}: Error finding container 06f5350f2bfaef7c34a47e047a2270fd58d005fc73ce53ceda4c4cdd37fb81c6: Status 404 returned error can't find the container with id 06f5350f2bfaef7c34a47e047a2270fd58d005fc73ce53ceda4c4cdd37fb81c6 Jul 16 12:46:23.590698 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:23.590661 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" event={"ID":"984f0cde-2733-42c7-b417-a2556ae98084","Type":"ContainerStarted","Data":"5981c4adc0f3ffc2f3f3e7dd0fc53ac017dcea53b3e79195acbafafba419c9b8"} Jul 16 12:46:23.591100 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:23.590741 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" event={"ID":"984f0cde-2733-42c7-b417-a2556ae98084","Type":"ContainerStarted","Data":"06f5350f2bfaef7c34a47e047a2270fd58d005fc73ce53ceda4c4cdd37fb81c6"} Jul 16 12:46:23.591100 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:23.590767 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" Jul 16 12:46:23.614716 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:23.614661 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" podStartSLOduration=1.6146478850000001 podStartE2EDuration="1.614647885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:46:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:46:23.612952524 +0000 UTC m=+436.846860376" watchObservedRunningTime="2026-07-16 12:46:23.614647885 +0000 UTC m=+436.848555735" Jul 16 12:46:23.638874 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:23.638845 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d" path="/var/lib/kubelet/pods/3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d/volumes" Jul 16 12:46:29.163792 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:29.163705 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:29.163792 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:29.163781 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:29.168503 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:29.168479 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:29.624880 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:29.624851 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-ccc56667d-725m6" Jul 16 12:46:30.575798 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:30.575770 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4vh8c" Jul 16 12:46:34.598127 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:34.598097 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" Jul 16 12:46:38.421128 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.421097 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw"] Jul 16 12:46:38.421519 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.421434 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d" containerName="manager" Jul 16 12:46:38.421519 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.421445 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d" containerName="manager" Jul 16 12:46:38.421519 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.421522 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bc42ec1-522b-4bc3-8ab9-ffade9c6a75d" containerName="manager" Jul 16 12:46:38.542628 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.542596 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw"] Jul 16 12:46:38.542771 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.542698 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.545356 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.545331 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 12:46:38.545489 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.545401 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-j9glc\"" Jul 16 12:46:38.600979 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.600948 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b139d0d7-91aa-4263-815e-2ed4038b0ae0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.601136 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.600991 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b139d0d7-91aa-4263-815e-2ed4038b0ae0-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.601136 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.601022 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b139d0d7-91aa-4263-815e-2ed4038b0ae0-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.601213 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.601138 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b139d0d7-91aa-4263-815e-2ed4038b0ae0-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.601213 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.601189 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b139d0d7-91aa-4263-815e-2ed4038b0ae0-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.601281 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.601213 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n4rq\" (UniqueName: \"kubernetes.io/projected/b139d0d7-91aa-4263-815e-2ed4038b0ae0-kube-api-access-8n4rq\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.601281 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.601234 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b139d0d7-91aa-4263-815e-2ed4038b0ae0-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.601281 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.601252 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b139d0d7-91aa-4263-815e-2ed4038b0ae0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.601388 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.601316 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b139d0d7-91aa-4263-815e-2ed4038b0ae0-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.702256 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.702231 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b139d0d7-91aa-4263-815e-2ed4038b0ae0-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.702386 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.702265 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b139d0d7-91aa-4263-815e-2ed4038b0ae0-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.702386 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.702286 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n4rq\" (UniqueName: \"kubernetes.io/projected/b139d0d7-91aa-4263-815e-2ed4038b0ae0-kube-api-access-8n4rq\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.702386 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.702317 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b139d0d7-91aa-4263-815e-2ed4038b0ae0-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.702386 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.702358 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b139d0d7-91aa-4263-815e-2ed4038b0ae0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.702586 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.702410 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b139d0d7-91aa-4263-815e-2ed4038b0ae0-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.702586 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.702473 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b139d0d7-91aa-4263-815e-2ed4038b0ae0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.702586 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.702517 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b139d0d7-91aa-4263-815e-2ed4038b0ae0-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.702586 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.702561 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b139d0d7-91aa-4263-815e-2ed4038b0ae0-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.702785 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.702647 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b139d0d7-91aa-4263-815e-2ed4038b0ae0-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.702940 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.702916 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b139d0d7-91aa-4263-815e-2ed4038b0ae0-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.703141 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.702947 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b139d0d7-91aa-4263-815e-2ed4038b0ae0-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.703141 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.703041 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b139d0d7-91aa-4263-815e-2ed4038b0ae0-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.703300 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.703265 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b139d0d7-91aa-4263-815e-2ed4038b0ae0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.704739 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.704713 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b139d0d7-91aa-4263-815e-2ed4038b0ae0-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.704947 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.704931 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b139d0d7-91aa-4263-815e-2ed4038b0ae0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.710595 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.710565 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b139d0d7-91aa-4263-815e-2ed4038b0ae0-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.710904 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.710886 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n4rq\" (UniqueName: \"kubernetes.io/projected/b139d0d7-91aa-4263-815e-2ed4038b0ae0-kube-api-access-8n4rq\") pod \"maas-default-gateway-openshift-default-56654dcf7-xq6gw\" (UID: \"b139d0d7-91aa-4263-815e-2ed4038b0ae0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:38.853943 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:38.853912 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:39.026700 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:39.025016 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw"] Jul 16 12:46:39.668813 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:39.668779 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" event={"ID":"b139d0d7-91aa-4263-815e-2ed4038b0ae0","Type":"ContainerStarted","Data":"f25c74d7fc826e5d9de00b132d1ae00077a25a8c6a227ac08cc2c553d42189bc"} Jul 16 12:46:42.470722 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:42.470682 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 12:46:42.471022 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:42.470765 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 12:46:42.471022 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:42.470795 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 12:46:42.686779 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:42.686746 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" event={"ID":"b139d0d7-91aa-4263-815e-2ed4038b0ae0","Type":"ContainerStarted","Data":"0acbc178ee15be0ea4cc41b0e81a6162b1b28c5c423203e9deca2b72b947c50f"} Jul 16 12:46:42.709701 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:42.709650 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" podStartSLOduration=1.268015029 podStartE2EDuration="4.709634217s" totalImagesPullingTime="3.441619188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:46:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:46:42.708147593 +0000 UTC m=+455.942055439" watchObservedRunningTime="2026-07-16 12:46:42.709634217 +0000 UTC m=+455.943542132" Jul 16 12:46:42.854097 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:42.854006 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:43.860281 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:43.860242 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:44.696781 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:44.696745 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:44.697858 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:44.697840 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:46:52.984542 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:52.984506 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:46:52.992593 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:52.992571 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-ltj7k" Jul 16 12:46:52.995216 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:52.995193 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 16 12:46:52.995627 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:52.995607 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:46:53.025765 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.025739 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:46:53.128639 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.128610 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/496c8746-66a2-433a-9c39-830fb301655a-config-file\") pod \"limitador-limitador-54d6d988d5-ltj7k\" (UID: \"496c8746-66a2-433a-9c39-830fb301655a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ltj7k" Jul 16 12:46:53.128775 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.128642 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcdtl\" (UniqueName: \"kubernetes.io/projected/496c8746-66a2-433a-9c39-830fb301655a-kube-api-access-lcdtl\") pod \"limitador-limitador-54d6d988d5-ltj7k\" (UID: \"496c8746-66a2-433a-9c39-830fb301655a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ltj7k" Jul 16 12:46:53.229575 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.229541 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/496c8746-66a2-433a-9c39-830fb301655a-config-file\") pod \"limitador-limitador-54d6d988d5-ltj7k\" (UID: \"496c8746-66a2-433a-9c39-830fb301655a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ltj7k" Jul 16 12:46:53.229575 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.229578 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcdtl\" (UniqueName: \"kubernetes.io/projected/496c8746-66a2-433a-9c39-830fb301655a-kube-api-access-lcdtl\") pod \"limitador-limitador-54d6d988d5-ltj7k\" (UID: \"496c8746-66a2-433a-9c39-830fb301655a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ltj7k" Jul 16 12:46:53.230207 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.230188 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/496c8746-66a2-433a-9c39-830fb301655a-config-file\") pod \"limitador-limitador-54d6d988d5-ltj7k\" (UID: \"496c8746-66a2-433a-9c39-830fb301655a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ltj7k" Jul 16 12:46:53.237526 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.237467 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcdtl\" (UniqueName: \"kubernetes.io/projected/496c8746-66a2-433a-9c39-830fb301655a-kube-api-access-lcdtl\") pod \"limitador-limitador-54d6d988d5-ltj7k\" (UID: \"496c8746-66a2-433a-9c39-830fb301655a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ltj7k" Jul 16 12:46:53.303890 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.303855 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-ltj7k" Jul 16 12:46:53.437913 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.437833 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:46:53.440192 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:46:53.440166 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod496c8746_66a2_433a_9c39_830fb301655a.slice/crio-9fa78d432b11d7296f37edd6aa154117c10c2a53b48f675da1ee5bef256a74ef WatchSource:0}: Error finding container 9fa78d432b11d7296f37edd6aa154117c10c2a53b48f675da1ee5bef256a74ef: Status 404 returned error can't find the container with id 9fa78d432b11d7296f37edd6aa154117c10c2a53b48f675da1ee5bef256a74ef Jul 16 12:46:53.578424 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.578351 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-tk8rg"] Jul 16 12:46:53.581413 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.581396 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tk8rg" Jul 16 12:46:53.585213 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.585190 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zgjb2\"" Jul 16 12:46:53.591113 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.591094 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tk8rg"] Jul 16 12:46:53.734229 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.734195 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmrgc\" (UniqueName: \"kubernetes.io/projected/b85dffdc-f129-496a-b4b3-7be789e7b83b-kube-api-access-zmrgc\") pod \"authorino-75d556bd77-tk8rg\" (UID: \"b85dffdc-f129-496a-b4b3-7be789e7b83b\") " pod="kuadrant-system/authorino-75d556bd77-tk8rg" Jul 16 12:46:53.743614 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.743583 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-ltj7k" event={"ID":"496c8746-66a2-433a-9c39-830fb301655a","Type":"ContainerStarted","Data":"9fa78d432b11d7296f37edd6aa154117c10c2a53b48f675da1ee5bef256a74ef"} Jul 16 12:46:53.834881 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.834802 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmrgc\" (UniqueName: \"kubernetes.io/projected/b85dffdc-f129-496a-b4b3-7be789e7b83b-kube-api-access-zmrgc\") pod \"authorino-75d556bd77-tk8rg\" (UID: \"b85dffdc-f129-496a-b4b3-7be789e7b83b\") " pod="kuadrant-system/authorino-75d556bd77-tk8rg" Jul 16 12:46:53.842917 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.842893 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmrgc\" (UniqueName: \"kubernetes.io/projected/b85dffdc-f129-496a-b4b3-7be789e7b83b-kube-api-access-zmrgc\") pod \"authorino-75d556bd77-tk8rg\" (UID: \"b85dffdc-f129-496a-b4b3-7be789e7b83b\") " pod="kuadrant-system/authorino-75d556bd77-tk8rg" Jul 16 12:46:53.890971 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:53.890945 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tk8rg" Jul 16 12:46:54.074600 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:54.074533 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tk8rg"] Jul 16 12:46:54.081640 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:46:54.081600 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb85dffdc_f129_496a_b4b3_7be789e7b83b.slice/crio-9202f2b28ead12a3f2a7aa098882acc604a1bf03e5be37e6d2217c7f1574cbab WatchSource:0}: Error finding container 9202f2b28ead12a3f2a7aa098882acc604a1bf03e5be37e6d2217c7f1574cbab: Status 404 returned error can't find the container with id 9202f2b28ead12a3f2a7aa098882acc604a1bf03e5be37e6d2217c7f1574cbab Jul 16 12:46:54.753297 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:54.753229 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tk8rg" event={"ID":"b85dffdc-f129-496a-b4b3-7be789e7b83b","Type":"ContainerStarted","Data":"9202f2b28ead12a3f2a7aa098882acc604a1bf03e5be37e6d2217c7f1574cbab"} Jul 16 12:46:58.776510 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:58.776467 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-ltj7k" event={"ID":"496c8746-66a2-433a-9c39-830fb301655a","Type":"ContainerStarted","Data":"821eae620694fe2e1f6a34246a4404283e693fb922a379c6a4f5d81d82e0f273"} Jul 16 12:46:58.776953 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:58.776527 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-ltj7k" Jul 16 12:46:58.777884 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:58.777859 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tk8rg" event={"ID":"b85dffdc-f129-496a-b4b3-7be789e7b83b","Type":"ContainerStarted","Data":"1af3138903ce601bc52d4a102e823ddd55060da332ceb2084b39dd25648ef1c4"} Jul 16 12:46:58.793533 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:58.793494 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-ltj7k" podStartSLOduration=2.349124817 podStartE2EDuration="6.793479977s" totalImagesPullingTime="4.44435516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:46:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:46:58.792783167 +0000 UTC m=+472.026691019" watchObservedRunningTime="2026-07-16 12:46:58.793479977 +0000 UTC m=+472.027387827" Jul 16 12:46:58.807818 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:46:58.807775 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-tk8rg" podStartSLOduration=2.005550946 podStartE2EDuration="5.807762812s" totalImagesPullingTime="3.802211866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:46:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:46:58.806658488 +0000 UTC m=+472.040566339" watchObservedRunningTime="2026-07-16 12:46:58.807762812 +0000 UTC m=+472.041670717" Jul 16 12:47:09.783651 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:09.783621 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-ltj7k" Jul 16 12:47:30.218373 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:30.218336 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tk8rg"] Jul 16 12:47:30.218880 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:30.218534 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-tk8rg" podUID="b85dffdc-f129-496a-b4b3-7be789e7b83b" containerName="authorino" containerID="cri-o://1af3138903ce601bc52d4a102e823ddd55060da332ceb2084b39dd25648ef1c4" gracePeriod=30 Jul 16 12:47:30.462875 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:30.462848 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tk8rg" Jul 16 12:47:30.540766 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:30.540698 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmrgc\" (UniqueName: \"kubernetes.io/projected/b85dffdc-f129-496a-b4b3-7be789e7b83b-kube-api-access-zmrgc\") pod \"b85dffdc-f129-496a-b4b3-7be789e7b83b\" (UID: \"b85dffdc-f129-496a-b4b3-7be789e7b83b\") " Jul 16 12:47:30.542684 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:30.542654 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b85dffdc-f129-496a-b4b3-7be789e7b83b-kube-api-access-zmrgc" (OuterVolumeSpecName: "kube-api-access-zmrgc") pod "b85dffdc-f129-496a-b4b3-7be789e7b83b" (UID: "b85dffdc-f129-496a-b4b3-7be789e7b83b"). InnerVolumeSpecName "kube-api-access-zmrgc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:47:30.641548 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:30.641522 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmrgc\" (UniqueName: \"kubernetes.io/projected/b85dffdc-f129-496a-b4b3-7be789e7b83b-kube-api-access-zmrgc\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:47:30.933239 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:30.933208 2567 generic.go:356] "Generic (PLEG): container finished" podID="b85dffdc-f129-496a-b4b3-7be789e7b83b" containerID="1af3138903ce601bc52d4a102e823ddd55060da332ceb2084b39dd25648ef1c4" exitCode=0 Jul 16 12:47:30.933405 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:30.933246 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tk8rg" Jul 16 12:47:30.933405 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:30.933249 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tk8rg" event={"ID":"b85dffdc-f129-496a-b4b3-7be789e7b83b","Type":"ContainerDied","Data":"1af3138903ce601bc52d4a102e823ddd55060da332ceb2084b39dd25648ef1c4"} Jul 16 12:47:30.933405 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:30.933291 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tk8rg" event={"ID":"b85dffdc-f129-496a-b4b3-7be789e7b83b","Type":"ContainerDied","Data":"9202f2b28ead12a3f2a7aa098882acc604a1bf03e5be37e6d2217c7f1574cbab"} Jul 16 12:47:30.933405 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:30.933308 2567 scope.go:117] "RemoveContainer" containerID="1af3138903ce601bc52d4a102e823ddd55060da332ceb2084b39dd25648ef1c4" Jul 16 12:47:30.942021 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:30.942004 2567 scope.go:117] "RemoveContainer" containerID="1af3138903ce601bc52d4a102e823ddd55060da332ceb2084b39dd25648ef1c4" Jul 16 12:47:30.942313 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:47:30.942296 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1af3138903ce601bc52d4a102e823ddd55060da332ceb2084b39dd25648ef1c4\": container with ID starting with 1af3138903ce601bc52d4a102e823ddd55060da332ceb2084b39dd25648ef1c4 not found: ID does not exist" containerID="1af3138903ce601bc52d4a102e823ddd55060da332ceb2084b39dd25648ef1c4" Jul 16 12:47:30.942383 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:30.942322 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1af3138903ce601bc52d4a102e823ddd55060da332ceb2084b39dd25648ef1c4"} err="failed to get container status \"1af3138903ce601bc52d4a102e823ddd55060da332ceb2084b39dd25648ef1c4\": rpc error: code = NotFound desc = could not find container \"1af3138903ce601bc52d4a102e823ddd55060da332ceb2084b39dd25648ef1c4\": container with ID starting with 1af3138903ce601bc52d4a102e823ddd55060da332ceb2084b39dd25648ef1c4 not found: ID does not exist" Jul 16 12:47:30.977231 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:30.977205 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tk8rg"] Jul 16 12:47:30.997481 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:30.997459 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tk8rg"] Jul 16 12:47:31.639384 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:47:31.639354 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b85dffdc-f129-496a-b4b3-7be789e7b83b" path="/var/lib/kubelet/pods/b85dffdc-f129-496a-b4b3-7be789e7b83b/volumes" Jul 16 12:48:11.490415 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:11.490340 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:48:18.941492 ip-10-0-140-165 kubenswrapper[2567]: E0716 12:48:18.941464 2567 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@0-14044-0.service\": RecentStats: unable to find data in memory cache]" Jul 16 12:48:19.173140 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:19.173039 2567 generic.go:356] "Generic (PLEG): container finished" podID="b139d0d7-91aa-4263-815e-2ed4038b0ae0" containerID="0acbc178ee15be0ea4cc41b0e81a6162b1b28c5c423203e9deca2b72b947c50f" exitCode=0 Jul 16 12:48:19.173140 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:19.173118 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" event={"ID":"b139d0d7-91aa-4263-815e-2ed4038b0ae0","Type":"ContainerDied","Data":"0acbc178ee15be0ea4cc41b0e81a6162b1b28c5c423203e9deca2b72b947c50f"} Jul 16 12:48:19.173507 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:19.173494 2567 scope.go:117] "RemoveContainer" containerID="0acbc178ee15be0ea4cc41b0e81a6162b1b28c5c423203e9deca2b72b947c50f" Jul 16 12:48:19.174487 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:19.174457 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 12:48:19.174567 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:19.174525 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 12:48:19.174567 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:19.174554 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 12:48:20.179770 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:20.179734 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" event={"ID":"b139d0d7-91aa-4263-815e-2ed4038b0ae0","Type":"ContainerStarted","Data":"b8c424c93c8552b3eee7161d7cf924dd1cd9dc6be878c1b3fe795b19cc0cdbca"} Jul 16 12:48:20.854907 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:20.854868 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:48:20.855193 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:20.855169 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xq6gw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 16 12:48:20.855292 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:20.855228 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" podUID="b139d0d7-91aa-4263-815e-2ed4038b0ae0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 16 12:48:21.854283 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:21.854247 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xq6gw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 16 12:48:21.854640 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:21.854321 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" podUID="b139d0d7-91aa-4263-815e-2ed4038b0ae0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 16 12:48:22.855146 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:22.855110 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xq6gw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 16 12:48:22.855525 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:22.855176 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" podUID="b139d0d7-91aa-4263-815e-2ed4038b0ae0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 16 12:48:23.855275 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:23.855240 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xq6gw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 16 12:48:23.855755 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:23.855293 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" podUID="b139d0d7-91aa-4263-815e-2ed4038b0ae0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 16 12:48:24.858189 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:24.858155 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:48:24.858590 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:24.858557 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:48:24.859215 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:24.859192 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xq6gw" Jul 16 12:48:38.188796 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:38.188760 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:48:54.077456 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:48:54.077420 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:49:04.787609 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:04.787569 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:49:07.180751 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:07.180710 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 12:49:07.181184 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:07.180731 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 12:49:07.184000 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:07.183982 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 12:49:07.184139 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:07.184004 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 12:49:14.393249 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.393215 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4"] Jul 16 12:49:14.393628 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.393579 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b85dffdc-f129-496a-b4b3-7be789e7b83b" containerName="authorino" Jul 16 12:49:14.393628 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.393590 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b85dffdc-f129-496a-b4b3-7be789e7b83b" containerName="authorino" Jul 16 12:49:14.393781 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.393654 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b85dffdc-f129-496a-b4b3-7be789e7b83b" containerName="authorino" Jul 16 12:49:14.395892 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.395872 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.399958 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.399937 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7jchv\"" Jul 16 12:49:14.400077 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.399991 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 16 12:49:14.400077 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.399939 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 16 12:49:14.400169 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.400090 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 16 12:49:14.407309 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.407286 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4"] Jul 16 12:49:14.422899 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.422867 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73e94106-9c1a-4809-bc51-cbaf135769af-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.423032 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.422987 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scbq8\" (UniqueName: \"kubernetes.io/projected/73e94106-9c1a-4809-bc51-cbaf135769af-kube-api-access-scbq8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.423149 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.423037 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73e94106-9c1a-4809-bc51-cbaf135769af-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.423149 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.423105 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73e94106-9c1a-4809-bc51-cbaf135769af-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.423149 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.423131 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73e94106-9c1a-4809-bc51-cbaf135769af-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.423300 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.423217 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73e94106-9c1a-4809-bc51-cbaf135769af-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.524271 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.524233 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73e94106-9c1a-4809-bc51-cbaf135769af-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.524271 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.524276 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scbq8\" (UniqueName: \"kubernetes.io/projected/73e94106-9c1a-4809-bc51-cbaf135769af-kube-api-access-scbq8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.524506 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.524303 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73e94106-9c1a-4809-bc51-cbaf135769af-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.524506 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.524333 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73e94106-9c1a-4809-bc51-cbaf135769af-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.524506 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.524348 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73e94106-9c1a-4809-bc51-cbaf135769af-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.524506 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.524388 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73e94106-9c1a-4809-bc51-cbaf135769af-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.524745 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.524693 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73e94106-9c1a-4809-bc51-cbaf135769af-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.524745 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.524711 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73e94106-9c1a-4809-bc51-cbaf135769af-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.524852 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.524794 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73e94106-9c1a-4809-bc51-cbaf135769af-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.527176 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.527143 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73e94106-9c1a-4809-bc51-cbaf135769af-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.527403 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.527379 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73e94106-9c1a-4809-bc51-cbaf135769af-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.533318 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.533293 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scbq8\" (UniqueName: \"kubernetes.io/projected/73e94106-9c1a-4809-bc51-cbaf135769af-kube-api-access-scbq8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4\" (UID: \"73e94106-9c1a-4809-bc51-cbaf135769af\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.708127 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.708092 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:14.844879 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:14.844855 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4"] Jul 16 12:49:14.847070 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:49:14.847028 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73e94106_9c1a_4809_bc51_cbaf135769af.slice/crio-6a6e24c43c9abb58d25a62fa253ceab788829fbbd7f329deb8d682ee506d246a WatchSource:0}: Error finding container 6a6e24c43c9abb58d25a62fa253ceab788829fbbd7f329deb8d682ee506d246a: Status 404 returned error can't find the container with id 6a6e24c43c9abb58d25a62fa253ceab788829fbbd7f329deb8d682ee506d246a Jul 16 12:49:15.447126 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:15.447087 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" event={"ID":"73e94106-9c1a-4809-bc51-cbaf135769af","Type":"ContainerStarted","Data":"6a6e24c43c9abb58d25a62fa253ceab788829fbbd7f329deb8d682ee506d246a"} Jul 16 12:49:16.187262 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:16.187230 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:49:16.453995 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:16.453907 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" event={"ID":"73e94106-9c1a-4809-bc51-cbaf135769af","Type":"ContainerStarted","Data":"e16d07da103ca08d3af008b165a46a19a46dbabd0226bee642bf217a76f3dd76"} Jul 16 12:49:16.479956 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:16.479908 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" podStartSLOduration=1.457829581 podStartE2EDuration="2.479891318s" totalImagesPullingTime="1.022061737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:49:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:49:16.477543037 +0000 UTC m=+609.711450887" watchObservedRunningTime="2026-07-16 12:49:16.479891318 +0000 UTC m=+609.713799168" Jul 16 12:49:22.377556 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.377519 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986"] Jul 16 12:49:22.380822 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.380804 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.383597 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.383576 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 16 12:49:22.391314 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.391293 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986"] Jul 16 12:49:22.491160 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.491128 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.491313 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.491168 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.491313 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.491187 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.491313 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.491261 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.491313 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.491285 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lhsm\" (UniqueName: \"kubernetes.io/projected/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-kube-api-access-8lhsm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.491467 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.491349 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.591941 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.591908 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.591941 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.591946 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.592147 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.591964 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.592147 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.592092 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.592147 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.592134 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lhsm\" (UniqueName: \"kubernetes.io/projected/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-kube-api-access-8lhsm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.592280 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.592203 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.592361 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.592335 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.592423 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.592392 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.592529 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.592511 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.594268 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.594251 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.594405 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.594389 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.600596 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.600575 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lhsm\" (UniqueName: \"kubernetes.io/projected/95fd15d5-9b3a-4cc4-98df-2faf116d5bfe-kube-api-access-8lhsm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pb986\" (UID: \"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.692598 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.692574 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:22.827954 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.827930 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986"] Jul 16 12:49:22.830143 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:49:22.830102 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95fd15d5_9b3a_4cc4_98df_2faf116d5bfe.slice/crio-b9031f2443c737447e5f807eb05a45e95c155e7fd0be6f8a6a12ba5954e7bd8d WatchSource:0}: Error finding container b9031f2443c737447e5f807eb05a45e95c155e7fd0be6f8a6a12ba5954e7bd8d: Status 404 returned error can't find the container with id b9031f2443c737447e5f807eb05a45e95c155e7fd0be6f8a6a12ba5954e7bd8d Jul 16 12:49:22.831635 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:22.831613 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 12:49:23.487273 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:23.487236 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" event={"ID":"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe","Type":"ContainerStarted","Data":"36dd18ca5a17c4f71a0040fbc628b773bfd32147e9858ffa81d8cf0cd90b3736"} Jul 16 12:49:23.487644 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:23.487291 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" event={"ID":"95fd15d5-9b3a-4cc4-98df-2faf116d5bfe","Type":"ContainerStarted","Data":"b9031f2443c737447e5f807eb05a45e95c155e7fd0be6f8a6a12ba5954e7bd8d"} Jul 16 12:49:23.507811 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:23.507758 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" podStartSLOduration=1.310426818 podStartE2EDuration="1.507739284s" totalImagesPullingTime="197.312466ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:49:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:49:23.507345152 +0000 UTC m=+616.741253005" watchObservedRunningTime="2026-07-16 12:49:23.507739284 +0000 UTC m=+616.741647138" Jul 16 12:49:23.879239 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:23.879158 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:49:24.708814 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:24.708780 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:24.709333 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:24.708844 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:24.723646 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:24.723623 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:25.508547 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:25.508514 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4" Jul 16 12:49:26.869393 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:26.869361 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z"] Jul 16 12:49:26.872290 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:26.872269 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:26.874884 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:26.874866 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 16 12:49:26.887942 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:26.887917 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z"] Jul 16 12:49:26.929290 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:26.929254 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47ea0e66-e55a-4612-92bd-5044973e8eb8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:26.929418 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:26.929393 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47ea0e66-e55a-4612-92bd-5044973e8eb8-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:26.929465 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:26.929423 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47ea0e66-e55a-4612-92bd-5044973e8eb8-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:26.929465 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:26.929447 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47ea0e66-e55a-4612-92bd-5044973e8eb8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:26.929543 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:26.929483 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47ea0e66-e55a-4612-92bd-5044973e8eb8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:26.929543 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:26.929499 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59fgp\" (UniqueName: \"kubernetes.io/projected/47ea0e66-e55a-4612-92bd-5044973e8eb8-kube-api-access-59fgp\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:27.030690 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:27.030652 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47ea0e66-e55a-4612-92bd-5044973e8eb8-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:27.030690 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:27.030684 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47ea0e66-e55a-4612-92bd-5044973e8eb8-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:27.030921 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:27.030704 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47ea0e66-e55a-4612-92bd-5044973e8eb8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:27.030921 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:27.030739 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47ea0e66-e55a-4612-92bd-5044973e8eb8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:27.030921 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:27.030863 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59fgp\" (UniqueName: \"kubernetes.io/projected/47ea0e66-e55a-4612-92bd-5044973e8eb8-kube-api-access-59fgp\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:27.031112 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:27.030928 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47ea0e66-e55a-4612-92bd-5044973e8eb8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:27.031112 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:27.031101 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47ea0e66-e55a-4612-92bd-5044973e8eb8-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:27.031235 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:27.031169 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47ea0e66-e55a-4612-92bd-5044973e8eb8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:27.031348 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:27.031324 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47ea0e66-e55a-4612-92bd-5044973e8eb8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:27.032979 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:27.032956 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47ea0e66-e55a-4612-92bd-5044973e8eb8-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:27.033263 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:27.033247 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47ea0e66-e55a-4612-92bd-5044973e8eb8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:27.039331 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:27.039305 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59fgp\" (UniqueName: \"kubernetes.io/projected/47ea0e66-e55a-4612-92bd-5044973e8eb8-kube-api-access-59fgp\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z\" (UID: \"47ea0e66-e55a-4612-92bd-5044973e8eb8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:27.182183 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:27.182160 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:27.316939 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:27.316915 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z"] Jul 16 12:49:27.319018 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:49:27.318993 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47ea0e66_e55a_4612_92bd_5044973e8eb8.slice/crio-e139d7d098b84e8469409af5f97b2c06ebb4b7ad7eea0e98646da4d89731568c WatchSource:0}: Error finding container e139d7d098b84e8469409af5f97b2c06ebb4b7ad7eea0e98646da4d89731568c: Status 404 returned error can't find the container with id e139d7d098b84e8469409af5f97b2c06ebb4b7ad7eea0e98646da4d89731568c Jul 16 12:49:27.508324 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:27.508290 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" event={"ID":"47ea0e66-e55a-4612-92bd-5044973e8eb8","Type":"ContainerStarted","Data":"e139d7d098b84e8469409af5f97b2c06ebb4b7ad7eea0e98646da4d89731568c"} Jul 16 12:49:28.514592 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:28.514555 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" event={"ID":"47ea0e66-e55a-4612-92bd-5044973e8eb8","Type":"ContainerStarted","Data":"7838bcd10fe74745c3f7b6f7da67588f5c8d15cab870f6e27b816517558ff2b1"} Jul 16 12:49:28.534953 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:28.534888 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" podStartSLOduration=2.361916704 podStartE2EDuration="2.534871937s" totalImagesPullingTime="172.955233ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:49:28.532316149 +0000 UTC m=+621.766224002" watchObservedRunningTime="2026-07-16 12:49:28.534871937 +0000 UTC m=+621.768779787" Jul 16 12:49:32.692857 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:32.692818 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:32.693298 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:32.692882 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:32.705310 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:32.705281 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:33.550900 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:33.550865 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pb986" Jul 16 12:49:35.183344 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:35.183314 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:49:37.183213 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:37.183166 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:37.183213 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:37.183216 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:37.195489 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:37.195466 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:49:37.569729 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:49:37.569642 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z" Jul 16 12:51:03.387517 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:51:03.387438 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:51:15.987310 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:51:15.987278 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:51:24.787997 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:51:24.787962 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:53:09.377785 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:53:09.377743 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:53:18.783630 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:53:18.783595 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:53:50.590090 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:53:50.590032 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:54:02.679586 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:54:02.679498 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:54:07.214947 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:54:07.214912 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 12:54:07.217100 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:54:07.217075 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 12:54:07.218267 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:54:07.218247 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 12:54:07.220662 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:54:07.220641 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 12:54:12.586311 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:54:12.586273 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:54:19.506252 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:54:19.506217 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:54:30.686127 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:54:30.686085 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:54:40.883526 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:54:40.883484 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:54:51.378467 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:54:51.378430 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:55:01.578550 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:55:01.578504 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:55:27.196881 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:55:27.196841 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:55:36.280702 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:55:36.280616 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:55:59.086601 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:55:59.086562 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:56:08.787732 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:56:08.787699 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:56:19.073112 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:56:19.073072 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:56:27.078376 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:56:27.078339 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:56:36.682568 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:56:36.682533 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:56:46.382998 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:56:46.382962 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:56:56.275440 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:56:56.275402 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:57:05.586721 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:05.586636 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:57:30.650322 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:30.650285 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259"] Jul 16 12:57:30.966915 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:30.966860 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" podUID="984f0cde-2733-42c7-b417-a2556ae98084" containerName="manager" containerID="cri-o://5981c4adc0f3ffc2f3f3e7dd0fc53ac017dcea53b3e79195acbafafba419c9b8" gracePeriod=10 Jul 16 12:57:32.979270 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:32.979232 2567 generic.go:356] "Generic (PLEG): container finished" podID="984f0cde-2733-42c7-b417-a2556ae98084" containerID="5981c4adc0f3ffc2f3f3e7dd0fc53ac017dcea53b3e79195acbafafba419c9b8" exitCode=0 Jul 16 12:57:32.979641 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:32.979307 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" event={"ID":"984f0cde-2733-42c7-b417-a2556ae98084","Type":"ContainerDied","Data":"5981c4adc0f3ffc2f3f3e7dd0fc53ac017dcea53b3e79195acbafafba419c9b8"} Jul 16 12:57:33.123828 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:33.123801 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" Jul 16 12:57:33.159198 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:33.159166 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jsh8\" (UniqueName: \"kubernetes.io/projected/984f0cde-2733-42c7-b417-a2556ae98084-kube-api-access-9jsh8\") pod \"984f0cde-2733-42c7-b417-a2556ae98084\" (UID: \"984f0cde-2733-42c7-b417-a2556ae98084\") " Jul 16 12:57:33.159385 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:33.159205 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/984f0cde-2733-42c7-b417-a2556ae98084-extensions-socket-volume\") pod \"984f0cde-2733-42c7-b417-a2556ae98084\" (UID: \"984f0cde-2733-42c7-b417-a2556ae98084\") " Jul 16 12:57:33.159609 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:33.159582 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/984f0cde-2733-42c7-b417-a2556ae98084-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "984f0cde-2733-42c7-b417-a2556ae98084" (UID: "984f0cde-2733-42c7-b417-a2556ae98084"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:57:33.161377 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:33.161344 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/984f0cde-2733-42c7-b417-a2556ae98084-kube-api-access-9jsh8" (OuterVolumeSpecName: "kube-api-access-9jsh8") pod "984f0cde-2733-42c7-b417-a2556ae98084" (UID: "984f0cde-2733-42c7-b417-a2556ae98084"). InnerVolumeSpecName "kube-api-access-9jsh8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:57:33.261000 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:33.260907 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jsh8\" (UniqueName: \"kubernetes.io/projected/984f0cde-2733-42c7-b417-a2556ae98084-kube-api-access-9jsh8\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:57:33.261000 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:33.260938 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/984f0cde-2733-42c7-b417-a2556ae98084-extensions-socket-volume\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 12:57:33.985648 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:33.985614 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" Jul 16 12:57:33.985648 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:33.985622 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259" event={"ID":"984f0cde-2733-42c7-b417-a2556ae98084","Type":"ContainerDied","Data":"06f5350f2bfaef7c34a47e047a2270fd58d005fc73ce53ceda4c4cdd37fb81c6"} Jul 16 12:57:33.986154 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:33.985681 2567 scope.go:117] "RemoveContainer" containerID="5981c4adc0f3ffc2f3f3e7dd0fc53ac017dcea53b3e79195acbafafba419c9b8" Jul 16 12:57:34.003949 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:34.003924 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259"] Jul 16 12:57:34.007038 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:34.007014 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ml259"] Jul 16 12:57:35.639186 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:57:35.639152 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="984f0cde-2733-42c7-b417-a2556ae98084" path="/var/lib/kubelet/pods/984f0cde-2733-42c7-b417-a2556ae98084/volumes" Jul 16 12:58:31.856169 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:31.856073 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbjrl"] Jul 16 12:58:31.856573 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:31.856452 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="984f0cde-2733-42c7-b417-a2556ae98084" containerName="manager" Jul 16 12:58:31.856573 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:31.856463 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="984f0cde-2733-42c7-b417-a2556ae98084" containerName="manager" Jul 16 12:58:31.856573 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:31.856547 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="984f0cde-2733-42c7-b417-a2556ae98084" containerName="manager" Jul 16 12:58:31.860729 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:31.860708 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbjrl" Jul 16 12:58:31.863247 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:31.863227 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nxk26\"" Jul 16 12:58:31.876141 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:31.876120 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbjrl"] Jul 16 12:58:31.978764 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:31.978722 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/279c83b0-aa6b-40dd-8d2c-804fd31182ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qbjrl\" (UID: \"279c83b0-aa6b-40dd-8d2c-804fd31182ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbjrl" Jul 16 12:58:31.978764 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:31.978765 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpvn2\" (UniqueName: \"kubernetes.io/projected/279c83b0-aa6b-40dd-8d2c-804fd31182ef-kube-api-access-mpvn2\") pod \"kuadrant-operator-controller-manager-86878c769f-qbjrl\" (UID: \"279c83b0-aa6b-40dd-8d2c-804fd31182ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbjrl" Jul 16 12:58:32.079990 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:32.079958 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/279c83b0-aa6b-40dd-8d2c-804fd31182ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qbjrl\" (UID: \"279c83b0-aa6b-40dd-8d2c-804fd31182ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbjrl" Jul 16 12:58:32.079990 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:32.079990 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpvn2\" (UniqueName: \"kubernetes.io/projected/279c83b0-aa6b-40dd-8d2c-804fd31182ef-kube-api-access-mpvn2\") pod \"kuadrant-operator-controller-manager-86878c769f-qbjrl\" (UID: \"279c83b0-aa6b-40dd-8d2c-804fd31182ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbjrl" Jul 16 12:58:32.080345 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:32.080324 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/279c83b0-aa6b-40dd-8d2c-804fd31182ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qbjrl\" (UID: \"279c83b0-aa6b-40dd-8d2c-804fd31182ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbjrl" Jul 16 12:58:32.088685 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:32.088655 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpvn2\" (UniqueName: \"kubernetes.io/projected/279c83b0-aa6b-40dd-8d2c-804fd31182ef-kube-api-access-mpvn2\") pod \"kuadrant-operator-controller-manager-86878c769f-qbjrl\" (UID: \"279c83b0-aa6b-40dd-8d2c-804fd31182ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbjrl" Jul 16 12:58:32.170752 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:32.170666 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbjrl" Jul 16 12:58:32.318713 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:32.318686 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbjrl"] Jul 16 12:58:32.320490 ip-10-0-140-165 kubenswrapper[2567]: W0716 12:58:32.320459 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod279c83b0_aa6b_40dd_8d2c_804fd31182ef.slice/crio-492708545c4329746a561c53d8f092df29c9dfc544353a50bf482dac0abf4a99 WatchSource:0}: Error finding container 492708545c4329746a561c53d8f092df29c9dfc544353a50bf482dac0abf4a99: Status 404 returned error can't find the container with id 492708545c4329746a561c53d8f092df29c9dfc544353a50bf482dac0abf4a99 Jul 16 12:58:33.287785 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:33.287747 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbjrl" event={"ID":"279c83b0-aa6b-40dd-8d2c-804fd31182ef","Type":"ContainerStarted","Data":"1f7ef9bce676129d6333f372fa6a287b21d06571b8c93e306322293553536a0d"} Jul 16 12:58:33.288201 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:33.287801 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbjrl" event={"ID":"279c83b0-aa6b-40dd-8d2c-804fd31182ef","Type":"ContainerStarted","Data":"492708545c4329746a561c53d8f092df29c9dfc544353a50bf482dac0abf4a99"} Jul 16 12:58:33.288201 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:33.287877 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbjrl" Jul 16 12:58:33.312868 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:33.312818 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbjrl" podStartSLOduration=2.312802457 podStartE2EDuration="2.312802457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:58:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:58:33.309863388 +0000 UTC m=+1166.543771238" watchObservedRunningTime="2026-07-16 12:58:33.312802457 +0000 UTC m=+1166.546710308" Jul 16 12:58:44.294719 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:44.294681 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbjrl" Jul 16 12:58:51.987272 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:58:51.987233 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:59:03.785983 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:59:03.785945 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:59:07.245505 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:59:07.245473 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 12:59:07.248684 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:59:07.248660 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 12:59:07.250572 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:59:07.250548 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 12:59:07.253200 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:59:07.253181 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 12:59:12.378197 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:59:12.378156 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:59:24.683817 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:59:24.683780 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:59:36.181457 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:59:36.181418 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:59:45.975119 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:59:45.975072 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 12:59:55.383488 ip-10-0-140-165 kubenswrapper[2567]: I0716 12:59:55.383446 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:00:06.578168 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:00:06.578088 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:00:17.285645 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:00:17.285612 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:00:26.485462 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:00:26.485425 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:00:42.791593 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:00:42.791543 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:01:01.126078 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:01:01.126018 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:01:08.486737 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:01:08.486697 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:01:17.880202 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:01:17.880162 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:01:28.471689 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:01:28.471651 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:01:48.882592 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:01:48.882550 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:01:58.078136 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:01:58.078098 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:02:07.575644 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:02:07.575610 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:02:17.676686 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:02:17.676643 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:02:26.884879 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:02:26.884833 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:02:36.283238 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:02:36.283204 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:02:49.874638 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:02:49.874602 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:02:58.780938 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:02:58.780897 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:03:08.280790 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:03:08.280744 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:03:19.779292 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:03:19.779255 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:03:31.277291 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:03:31.277251 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:03:40.285731 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:03:40.285690 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:03:51.278968 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:03:51.278933 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:04:03.177606 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:04:03.177566 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:04:07.274782 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:04:07.274745 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 13:04:07.277859 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:04:07.277835 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 13:04:07.280589 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:04:07.280567 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 13:04:07.283445 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:04:07.283429 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 13:04:16.674966 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:04:16.674930 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:04:33.074323 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:04:33.074288 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:04:42.379084 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:04:42.379036 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:04:58.181075 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:04:58.181024 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:05:09.679895 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:05:09.679857 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:06:29.682527 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:06:29.682434 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:06:39.774388 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:06:39.774344 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:06:51.881594 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:06:51.881551 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:07:07.481569 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:07:07.481525 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:07:29.280188 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:07:29.280101 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:07:42.383752 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:07:42.383712 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:08:31.876999 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:08:31.876962 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:08:44.772281 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:08:44.772240 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:09:02.371110 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:02.371011 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:09:07.303251 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:07.303221 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 13:09:07.306303 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:07.306283 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 13:09:07.310455 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:07.310426 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 13:09:07.313391 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:07.313373 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 13:09:13.785165 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:13.785127 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:09:19.581859 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.581830 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj"] Jul 16 13:09:19.585238 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.585221 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.590855 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.590818 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-cb4eee\"/\"test-model-cb4eee-kserve-self-signed-certs\"" Jul 16 13:09:19.591065 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.590833 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-cb4eee\"/\"kube-root-ca.crt\"" Jul 16 13:09:19.591241 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.590884 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-cb4eee\"/\"openshift-service-ca.crt\"" Jul 16 13:09:19.591475 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.590906 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-cb4eee\"/\"default-dockercfg-zpzfl\"" Jul 16 13:09:19.596267 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.596248 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj"] Jul 16 13:09:19.687175 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.687147 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-dshm\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.687287 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.687177 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-model-cache\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.687287 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.687206 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-home\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.687287 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.687275 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dc4h\" (UniqueName: \"kubernetes.io/projected/bc5d14d6-b0ec-46ad-880a-d83090e3643d-kube-api-access-6dc4h\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.687400 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.687323 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc5d14d6-b0ec-46ad-880a-d83090e3643d-tls-certs\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.687400 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.687378 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-tmp-dir\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.788400 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.788371 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-tmp-dir\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.788542 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.788409 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-dshm\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.788542 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.788431 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-model-cache\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.788660 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.788593 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-home\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.788718 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.788661 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dc4h\" (UniqueName: \"kubernetes.io/projected/bc5d14d6-b0ec-46ad-880a-d83090e3643d-kube-api-access-6dc4h\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.788788 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.788765 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc5d14d6-b0ec-46ad-880a-d83090e3643d-tls-certs\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.788848 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.788796 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-model-cache\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.788848 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.788811 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-tmp-dir\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.788973 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.788951 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-home\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.790721 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.790672 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-dshm\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.791161 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.791142 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc5d14d6-b0ec-46ad-880a-d83090e3643d-tls-certs\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.797202 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.797178 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dc4h\" (UniqueName: \"kubernetes.io/projected/bc5d14d6-b0ec-46ad-880a-d83090e3643d-kube-api-access-6dc4h\") pod \"test-model-cb4eee-kserve-7b74c8b4cc-66qxj\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:19.895911 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:19.895835 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:20.029283 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:20.029082 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj"] Jul 16 13:09:20.031714 ip-10-0-140-165 kubenswrapper[2567]: W0716 13:09:20.031688 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc5d14d6_b0ec_46ad_880a_d83090e3643d.slice/crio-ae9b46ba3d0b46b8449e3d14d222395ca7ae2bd00e7ff52ffba6846f7b6c48aa WatchSource:0}: Error finding container ae9b46ba3d0b46b8449e3d14d222395ca7ae2bd00e7ff52ffba6846f7b6c48aa: Status 404 returned error can't find the container with id ae9b46ba3d0b46b8449e3d14d222395ca7ae2bd00e7ff52ffba6846f7b6c48aa Jul 16 13:09:20.579357 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:20.579316 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" event={"ID":"bc5d14d6-b0ec-46ad-880a-d83090e3643d","Type":"ContainerStarted","Data":"5e30083b8f78e81f7cd0f964c9cc173cd6239dfe1d35b5a794567d7b0f07ac97"} Jul 16 13:09:20.579539 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:20.579363 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" event={"ID":"bc5d14d6-b0ec-46ad-880a-d83090e3643d","Type":"ContainerStarted","Data":"ae9b46ba3d0b46b8449e3d14d222395ca7ae2bd00e7ff52ffba6846f7b6c48aa"} Jul 16 13:09:20.596962 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:20.596918 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" podStartSLOduration=1.596904009 podStartE2EDuration="1.596904009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 13:09:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 13:09:20.595451582 +0000 UTC m=+1813.829359433" watchObservedRunningTime="2026-07-16 13:09:20.596904009 +0000 UTC m=+1813.830811860" Jul 16 13:09:28.176396 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:28.176361 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:09:29.896939 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:29.896899 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:29.897354 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:29.896963 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:29.909686 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:29.909659 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:30.642286 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:30.642251 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:40.178538 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:40.178504 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj"] Jul 16 13:09:47.483783 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:47.483751 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:09:55.198969 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.198898 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" podUID="bc5d14d6-b0ec-46ad-880a-d83090e3643d" containerName="main" containerID="cri-o://5e30083b8f78e81f7cd0f964c9cc173cd6239dfe1d35b5a794567d7b0f07ac97" gracePeriod=45 Jul 16 13:09:55.465428 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.465401 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:55.577917 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.577894 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-model-cache\") pod \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " Jul 16 13:09:55.578093 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.577924 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-home\") pod \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " Jul 16 13:09:55.578093 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.577965 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-tmp-dir\") pod \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " Jul 16 13:09:55.578093 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.578005 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dc4h\" (UniqueName: \"kubernetes.io/projected/bc5d14d6-b0ec-46ad-880a-d83090e3643d-kube-api-access-6dc4h\") pod \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " Jul 16 13:09:55.578256 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.578119 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-dshm\") pod \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " Jul 16 13:09:55.578256 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.578144 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc5d14d6-b0ec-46ad-880a-d83090e3643d-tls-certs\") pod \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\" (UID: \"bc5d14d6-b0ec-46ad-880a-d83090e3643d\") " Jul 16 13:09:55.578256 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.578239 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-home" (OuterVolumeSpecName: "home") pod "bc5d14d6-b0ec-46ad-880a-d83090e3643d" (UID: "bc5d14d6-b0ec-46ad-880a-d83090e3643d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 13:09:55.578414 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.578261 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-model-cache" (OuterVolumeSpecName: "model-cache") pod "bc5d14d6-b0ec-46ad-880a-d83090e3643d" (UID: "bc5d14d6-b0ec-46ad-880a-d83090e3643d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 13:09:55.578414 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.578249 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bc5d14d6-b0ec-46ad-880a-d83090e3643d" (UID: "bc5d14d6-b0ec-46ad-880a-d83090e3643d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 13:09:55.578508 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.578449 2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 13:09:55.578508 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.578462 2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 13:09:55.578508 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.578471 2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 13:09:55.580483 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.580446 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc5d14d6-b0ec-46ad-880a-d83090e3643d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bc5d14d6-b0ec-46ad-880a-d83090e3643d" (UID: "bc5d14d6-b0ec-46ad-880a-d83090e3643d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 13:09:55.580483 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.580461 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-dshm" (OuterVolumeSpecName: "dshm") pod "bc5d14d6-b0ec-46ad-880a-d83090e3643d" (UID: "bc5d14d6-b0ec-46ad-880a-d83090e3643d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 13:09:55.580626 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.580502 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc5d14d6-b0ec-46ad-880a-d83090e3643d-kube-api-access-6dc4h" (OuterVolumeSpecName: "kube-api-access-6dc4h") pod "bc5d14d6-b0ec-46ad-880a-d83090e3643d" (UID: "bc5d14d6-b0ec-46ad-880a-d83090e3643d"). InnerVolumeSpecName "kube-api-access-6dc4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 13:09:55.679069 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.679025 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dc4h\" (UniqueName: \"kubernetes.io/projected/bc5d14d6-b0ec-46ad-880a-d83090e3643d-kube-api-access-6dc4h\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 13:09:55.679174 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.679070 2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc5d14d6-b0ec-46ad-880a-d83090e3643d-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 13:09:55.679174 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.679085 2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc5d14d6-b0ec-46ad-880a-d83090e3643d-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 13:09:55.751213 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.751129 2567 generic.go:356] "Generic (PLEG): container finished" podID="bc5d14d6-b0ec-46ad-880a-d83090e3643d" containerID="5e30083b8f78e81f7cd0f964c9cc173cd6239dfe1d35b5a794567d7b0f07ac97" exitCode=0 Jul 16 13:09:55.751213 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.751165 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" Jul 16 13:09:55.751213 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.751200 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" event={"ID":"bc5d14d6-b0ec-46ad-880a-d83090e3643d","Type":"ContainerDied","Data":"5e30083b8f78e81f7cd0f964c9cc173cd6239dfe1d35b5a794567d7b0f07ac97"} Jul 16 13:09:55.751445 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.751240 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj" event={"ID":"bc5d14d6-b0ec-46ad-880a-d83090e3643d","Type":"ContainerDied","Data":"ae9b46ba3d0b46b8449e3d14d222395ca7ae2bd00e7ff52ffba6846f7b6c48aa"} Jul 16 13:09:55.751445 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.751261 2567 scope.go:117] "RemoveContainer" containerID="5e30083b8f78e81f7cd0f964c9cc173cd6239dfe1d35b5a794567d7b0f07ac97" Jul 16 13:09:55.760462 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.760447 2567 scope.go:117] "RemoveContainer" containerID="5e30083b8f78e81f7cd0f964c9cc173cd6239dfe1d35b5a794567d7b0f07ac97" Jul 16 13:09:55.760708 ip-10-0-140-165 kubenswrapper[2567]: E0716 13:09:55.760691 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e30083b8f78e81f7cd0f964c9cc173cd6239dfe1d35b5a794567d7b0f07ac97\": container with ID starting with 5e30083b8f78e81f7cd0f964c9cc173cd6239dfe1d35b5a794567d7b0f07ac97 not found: ID does not exist" containerID="5e30083b8f78e81f7cd0f964c9cc173cd6239dfe1d35b5a794567d7b0f07ac97" Jul 16 13:09:55.760755 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.760724 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e30083b8f78e81f7cd0f964c9cc173cd6239dfe1d35b5a794567d7b0f07ac97"} err="failed to get container status \"5e30083b8f78e81f7cd0f964c9cc173cd6239dfe1d35b5a794567d7b0f07ac97\": rpc error: code = NotFound desc = could not find container \"5e30083b8f78e81f7cd0f964c9cc173cd6239dfe1d35b5a794567d7b0f07ac97\": container with ID starting with 5e30083b8f78e81f7cd0f964c9cc173cd6239dfe1d35b5a794567d7b0f07ac97 not found: ID does not exist" Jul 16 13:09:55.767685 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.767663 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj"] Jul 16 13:09:55.771273 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:55.771248 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-cb4eee/test-model-cb4eee-kserve-7b74c8b4cc-66qxj"] Jul 16 13:09:57.640336 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:09:57.640302 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc5d14d6-b0ec-46ad-880a-d83090e3643d" path="/var/lib/kubelet/pods/bc5d14d6-b0ec-46ad-880a-d83090e3643d/volumes" Jul 16 13:10:27.074019 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:10:27.073981 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:10:39.677941 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:10:39.677861 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:10:54.786618 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:10:54.786581 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:11:11.778168 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:11.778137 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:11:25.882336 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:25.882306 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:11:32.054967 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.054937 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46"] Jul 16 13:11:32.055454 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.055437 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5d14d6-b0ec-46ad-880a-d83090e3643d" containerName="main" Jul 16 13:11:32.055500 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.055456 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5d14d6-b0ec-46ad-880a-d83090e3643d" containerName="main" Jul 16 13:11:32.055548 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.055530 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc5d14d6-b0ec-46ad-880a-d83090e3643d" containerName="main" Jul 16 13:11:32.058734 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.058716 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46" Jul 16 13:11:32.062463 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.062435 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-8d69f8-serving-cert\"" Jul 16 13:11:32.062595 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.062440 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 16 13:11:32.063616 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.063594 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dgzx9\"" Jul 16 13:11:32.063741 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.063620 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 13:11:32.071459 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.071437 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46"] Jul 16 13:11:32.209084 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.209035 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b04e5aa7-a1cf-4b80-973e-0d198480a39c-maas-api-tls\") pod \"maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46\" (UID: \"b04e5aa7-a1cf-4b80-973e-0d198480a39c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46" Jul 16 13:11:32.209084 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.209093 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cln8\" (UniqueName: \"kubernetes.io/projected/b04e5aa7-a1cf-4b80-973e-0d198480a39c-kube-api-access-9cln8\") pod \"maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46\" (UID: \"b04e5aa7-a1cf-4b80-973e-0d198480a39c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46" Jul 16 13:11:32.209306 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.209139 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b04e5aa7-a1cf-4b80-973e-0d198480a39c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46\" (UID: \"b04e5aa7-a1cf-4b80-973e-0d198480a39c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46" Jul 16 13:11:32.310027 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.309932 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cln8\" (UniqueName: \"kubernetes.io/projected/b04e5aa7-a1cf-4b80-973e-0d198480a39c-kube-api-access-9cln8\") pod \"maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46\" (UID: \"b04e5aa7-a1cf-4b80-973e-0d198480a39c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46" Jul 16 13:11:32.310027 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.309993 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b04e5aa7-a1cf-4b80-973e-0d198480a39c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46\" (UID: \"b04e5aa7-a1cf-4b80-973e-0d198480a39c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46" Jul 16 13:11:32.310285 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.310151 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b04e5aa7-a1cf-4b80-973e-0d198480a39c-maas-api-tls\") pod \"maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46\" (UID: \"b04e5aa7-a1cf-4b80-973e-0d198480a39c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46" Jul 16 13:11:32.310680 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.310654 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b04e5aa7-a1cf-4b80-973e-0d198480a39c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46\" (UID: \"b04e5aa7-a1cf-4b80-973e-0d198480a39c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46" Jul 16 13:11:32.312608 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.312581 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b04e5aa7-a1cf-4b80-973e-0d198480a39c-maas-api-tls\") pod \"maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46\" (UID: \"b04e5aa7-a1cf-4b80-973e-0d198480a39c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46" Jul 16 13:11:32.318850 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.318826 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cln8\" (UniqueName: \"kubernetes.io/projected/b04e5aa7-a1cf-4b80-973e-0d198480a39c-kube-api-access-9cln8\") pod \"maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46\" (UID: \"b04e5aa7-a1cf-4b80-973e-0d198480a39c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46" Jul 16 13:11:32.372485 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.372454 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46" Jul 16 13:11:32.512077 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.512036 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46"] Jul 16 13:11:32.514000 ip-10-0-140-165 kubenswrapper[2567]: W0716 13:11:32.513975 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb04e5aa7_a1cf_4b80_973e_0d198480a39c.slice/crio-d4954fac125cc42b9341b78dd560ef4a712daba7cdbb7f8aea86db2bc16b2575 WatchSource:0}: Error finding container d4954fac125cc42b9341b78dd560ef4a712daba7cdbb7f8aea86db2bc16b2575: Status 404 returned error can't find the container with id d4954fac125cc42b9341b78dd560ef4a712daba7cdbb7f8aea86db2bc16b2575 Jul 16 13:11:32.515465 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:32.515446 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 13:11:33.244707 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:33.244672 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46" event={"ID":"b04e5aa7-a1cf-4b80-973e-0d198480a39c","Type":"ContainerStarted","Data":"d4954fac125cc42b9341b78dd560ef4a712daba7cdbb7f8aea86db2bc16b2575"} Jul 16 13:11:38.272699 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:38.272663 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46" event={"ID":"b04e5aa7-a1cf-4b80-973e-0d198480a39c","Type":"ContainerStarted","Data":"05d3caf664b5fbf7b0466b0f45acb881582e6840473a6f4b6243c92e3b96b523"} Jul 16 13:11:38.273113 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:38.272739 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46" Jul 16 13:11:38.293387 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:38.293344 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46" podStartSLOduration=1.306444711 podStartE2EDuration="6.293331105s" totalImagesPullingTime="4.986886394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 13:11:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 13:11:38.291703899 +0000 UTC m=+1951.525611751" watchObservedRunningTime="2026-07-16 13:11:38.293331105 +0000 UTC m=+1951.527238997" Jul 16 13:11:42.435658 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:42.435616 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:11:44.283043 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:11:44.283016 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46" Jul 16 13:12:03.594360 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:12:03.594279 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:12:34.991633 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:12:34.991597 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:12:51.387085 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:12:51.387038 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:13:11.383017 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:13:11.382981 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:13:26.585606 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:13:26.585572 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:13:45.288610 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:13:45.288524 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:14:05.412292 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:14:05.412255 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:14:07.341649 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:14:07.341616 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 13:14:07.344877 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:14:07.344851 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 13:14:07.350883 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:14:07.350865 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 13:14:07.353654 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:14:07.353636 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 13:14:27.687699 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:14:27.687665 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:14:47.883565 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:14:47.883531 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:15:00.141107 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.140993 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-cb4eee-29736795-4d6nf"] Jul 16 13:15:00.143601 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.143579 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ae18e9-29736795-rqqwf"] Jul 16 13:15:00.143739 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.143706 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-cb4eee-29736795-4d6nf" Jul 16 13:15:00.145718 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.145700 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-41581b25-29736795-czpgh"] Jul 16 13:15:00.145859 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.145835 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ae18e9-29736795-rqqwf" Jul 16 13:15:00.147578 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.147559 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5a470a1b-29736795-w4kff"] Jul 16 13:15:00.147774 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.147750 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-41581b25-29736795-czpgh" Jul 16 13:15:00.149697 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.149681 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-b0d8b7c2-29736795-kr82m"] Jul 16 13:15:00.149799 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.149783 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5a470a1b-29736795-w4kff" Jul 16 13:15:00.150191 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.150174 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zhdlk\"" Jul 16 13:15:00.152153 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.152137 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-b0d8b7c2-29736795-kr82m" Jul 16 13:15:00.154404 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.154386 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-cb4eee-29736795-4d6nf" Jul 16 13:15:00.165335 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.165307 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ae18e9-29736795-rqqwf" Jul 16 13:15:00.165954 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.165922 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-cb4eee-29736795-4d6nf"] Jul 16 13:15:00.168280 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.168238 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ae18e9-29736795-rqqwf"] Jul 16 13:15:00.181613 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.181591 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-213b02-29736795-gzxrq"] Jul 16 13:15:00.187590 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.186633 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-a22377-29736795-pwv2d"] Jul 16 13:15:00.187590 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.186768 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-213b02-29736795-gzxrq" Jul 16 13:15:00.191980 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.190315 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-60a47609-29736795-h545g"] Jul 16 13:15:00.192575 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.192554 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-a22377-29736795-pwv2d" Jul 16 13:15:00.194228 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.194206 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b12c92-29736795-swrqg"] Jul 16 13:15:00.196744 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.196632 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-60a47609-29736795-h545g" Jul 16 13:15:00.199665 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.199456 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-b0d8b7c2-29736795-kr82m"] Jul 16 13:15:00.199665 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.199552 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b12c92-29736795-swrqg" Jul 16 13:15:00.206078 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.205721 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5a470a1b-29736795-w4kff"] Jul 16 13:15:00.207633 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.207578 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-41581b25-29736795-czpgh"] Jul 16 13:15:00.210499 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.210349 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-41581b25-29736795-czpgh" Jul 16 13:15:00.212994 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.212972 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-213b02-29736795-gzxrq"] Jul 16 13:15:00.215579 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.215535 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-a22377-29736795-pwv2d"] Jul 16 13:15:00.220022 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.219803 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b12c92-29736795-swrqg"] Jul 16 13:15:00.222301 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.222122 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5a470a1b-29736795-w4kff" Jul 16 13:15:00.241126 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.238694 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-60a47609-29736795-h545g"] Jul 16 13:15:00.291581 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.291550 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-b0d8b7c2-29736795-kr82m" Jul 16 13:15:00.303311 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.303127 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-213b02-29736795-gzxrq" Jul 16 13:15:00.331934 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.331379 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-a22377-29736795-pwv2d" Jul 16 13:15:00.362645 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.362399 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-60a47609-29736795-h545g" Jul 16 13:15:00.380741 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.380709 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b12c92-29736795-swrqg" Jul 16 13:15:00.615272 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.615244 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-cb4eee-29736795-4d6nf"] Jul 16 13:15:00.621691 ip-10-0-140-165 kubenswrapper[2567]: W0716 13:15:00.621667 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e8aad7b_5765_4b99_9e0f_ac6332d17250.slice/crio-b6c032aa2cb0d1107a6726f49c09498eb4481c32ecb7c9f817d4c5b8b828df5e WatchSource:0}: Error finding container b6c032aa2cb0d1107a6726f49c09498eb4481c32ecb7c9f817d4c5b8b828df5e: Status 404 returned error can't find the container with id b6c032aa2cb0d1107a6726f49c09498eb4481c32ecb7c9f817d4c5b8b828df5e Jul 16 13:15:00.643863 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.643796 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-a22377-29736795-pwv2d"] Jul 16 13:15:00.645805 ip-10-0-140-165 kubenswrapper[2567]: W0716 13:15:00.645774 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d256401_fd4a_410b_8330_4ed55cc4054d.slice/crio-80f81d3f319d1061e046600d85f5a5f8ceef9dbf9335c963fadfb929f6e715bd WatchSource:0}: Error finding container 80f81d3f319d1061e046600d85f5a5f8ceef9dbf9335c963fadfb929f6e715bd: Status 404 returned error can't find the container with id 80f81d3f319d1061e046600d85f5a5f8ceef9dbf9335c963fadfb929f6e715bd Jul 16 13:15:00.662634 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.662610 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ae18e9-29736795-rqqwf"] Jul 16 13:15:00.664107 ip-10-0-140-165 kubenswrapper[2567]: W0716 13:15:00.664080 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod846fe269_bf8f_4b63_89ed_b44fa0fbb2c1.slice/crio-f40211c12f504d054ccbc3f0e1d2bf80a8f101e2d885092046d5e71cc7bd7563 WatchSource:0}: Error finding container f40211c12f504d054ccbc3f0e1d2bf80a8f101e2d885092046d5e71cc7bd7563: Status 404 returned error can't find the container with id f40211c12f504d054ccbc3f0e1d2bf80a8f101e2d885092046d5e71cc7bd7563 Jul 16 13:15:00.687665 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.687645 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-41581b25-29736795-czpgh"] Jul 16 13:15:00.689122 ip-10-0-140-165 kubenswrapper[2567]: W0716 13:15:00.689103 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fcb1d30_c0d8_4979_a128_336a8b88c32b.slice/crio-59e10da43374ed02ecbb0b6f9fa8cfe61750b5f9969e6ffb4ce94922ade0457a WatchSource:0}: Error finding container 59e10da43374ed02ecbb0b6f9fa8cfe61750b5f9969e6ffb4ce94922ade0457a: Status 404 returned error can't find the container with id 59e10da43374ed02ecbb0b6f9fa8cfe61750b5f9969e6ffb4ce94922ade0457a Jul 16 13:15:00.730400 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.729384 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5a470a1b-29736795-w4kff"] Jul 16 13:15:00.730743 ip-10-0-140-165 kubenswrapper[2567]: W0716 13:15:00.730714 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf57401d_ce7c_4de5_a254_3cf5f8061cf6.slice/crio-eb7ebad96f3ad0153e20026d9c7c1779a84e2cf211171c69cf2642bfc4a212b1 WatchSource:0}: Error finding container eb7ebad96f3ad0153e20026d9c7c1779a84e2cf211171c69cf2642bfc4a212b1: Status 404 returned error can't find the container with id eb7ebad96f3ad0153e20026d9c7c1779a84e2cf211171c69cf2642bfc4a212b1 Jul 16 13:15:00.811620 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.811599 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-b0d8b7c2-29736795-kr82m"] Jul 16 13:15:00.813991 ip-10-0-140-165 kubenswrapper[2567]: W0716 13:15:00.813951 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9002ea6_c837_4845_b0a7_622e5073c21a.slice/crio-d53f221e6d6ddd8d9d4b3259f2b09a55d21866f685fe45e95d48a3f003c0dece WatchSource:0}: Error finding container d53f221e6d6ddd8d9d4b3259f2b09a55d21866f685fe45e95d48a3f003c0dece: Status 404 returned error can't find the container with id d53f221e6d6ddd8d9d4b3259f2b09a55d21866f685fe45e95d48a3f003c0dece Jul 16 13:15:00.820364 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.820340 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-213b02-29736795-gzxrq"] Jul 16 13:15:00.821229 ip-10-0-140-165 kubenswrapper[2567]: W0716 13:15:00.821209 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod692cd48f_bbaf_47d8_be8c_6ec3b690fe31.slice/crio-561d06b23d795b17245566c28a497cd02b14855c7db8e916b9b1fd31fae45211 WatchSource:0}: Error finding container 561d06b23d795b17245566c28a497cd02b14855c7db8e916b9b1fd31fae45211: Status 404 returned error can't find the container with id 561d06b23d795b17245566c28a497cd02b14855c7db8e916b9b1fd31fae45211 Jul 16 13:15:00.875783 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.875730 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-60a47609-29736795-h545g"] Jul 16 13:15:00.878308 ip-10-0-140-165 kubenswrapper[2567]: W0716 13:15:00.878284 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb22a060_aa2f_4036_aab5_bd59453d59b8.slice/crio-80efc2ad9c97e36ba3c67a4386197be317f00329c15fe02788b6624c6a33cbc8 WatchSource:0}: Error finding container 80efc2ad9c97e36ba3c67a4386197be317f00329c15fe02788b6624c6a33cbc8: Status 404 returned error can't find the container with id 80efc2ad9c97e36ba3c67a4386197be317f00329c15fe02788b6624c6a33cbc8 Jul 16 13:15:00.879070 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:00.878943 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b12c92-29736795-swrqg"] Jul 16 13:15:00.881463 ip-10-0-140-165 kubenswrapper[2567]: W0716 13:15:00.881443 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9d1d95e_a3bc_4550_860f_f496b4ece340.slice/crio-c9e5d4856a4c0bcafccf968e2135d6ff8ef4e8e64b68d199895500b534e9d9fb WatchSource:0}: Error finding container c9e5d4856a4c0bcafccf968e2135d6ff8ef4e8e64b68d199895500b534e9d9fb: Status 404 returned error can't find the container with id c9e5d4856a4c0bcafccf968e2135d6ff8ef4e8e64b68d199895500b534e9d9fb Jul 16 13:15:01.283251 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:01.283217 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-a22377-29736795-pwv2d" event={"ID":"2d256401-fd4a-410b-8330-4ed55cc4054d","Type":"ContainerStarted","Data":"80f81d3f319d1061e046600d85f5a5f8ceef9dbf9335c963fadfb929f6e715bd"} Jul 16 13:15:01.284477 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:01.284452 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5a470a1b-29736795-w4kff" event={"ID":"bf57401d-ce7c-4de5-a254-3cf5f8061cf6","Type":"ContainerStarted","Data":"eb7ebad96f3ad0153e20026d9c7c1779a84e2cf211171c69cf2642bfc4a212b1"} Jul 16 13:15:01.285660 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:01.285638 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-b0d8b7c2-29736795-kr82m" event={"ID":"f9002ea6-c837-4845-b0a7-622e5073c21a","Type":"ContainerStarted","Data":"d53f221e6d6ddd8d9d4b3259f2b09a55d21866f685fe45e95d48a3f003c0dece"} Jul 16 13:15:01.286780 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:01.286758 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-cb4eee-29736795-4d6nf" event={"ID":"6e8aad7b-5765-4b99-9e0f-ac6332d17250","Type":"ContainerStarted","Data":"b6c032aa2cb0d1107a6726f49c09498eb4481c32ecb7c9f817d4c5b8b828df5e"} Jul 16 13:15:01.288015 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:01.287977 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b12c92-29736795-swrqg" event={"ID":"c9d1d95e-a3bc-4550-860f-f496b4ece340","Type":"ContainerStarted","Data":"c9e5d4856a4c0bcafccf968e2135d6ff8ef4e8e64b68d199895500b534e9d9fb"} Jul 16 13:15:01.289271 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:01.289248 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-213b02-29736795-gzxrq" event={"ID":"692cd48f-bbaf-47d8-be8c-6ec3b690fe31","Type":"ContainerStarted","Data":"561d06b23d795b17245566c28a497cd02b14855c7db8e916b9b1fd31fae45211"} Jul 16 13:15:01.290309 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:01.290289 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ae18e9-29736795-rqqwf" event={"ID":"846fe269-bf8f-4b63-89ed-b44fa0fbb2c1","Type":"ContainerStarted","Data":"f40211c12f504d054ccbc3f0e1d2bf80a8f101e2d885092046d5e71cc7bd7563"} Jul 16 13:15:01.291351 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:01.291330 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-60a47609-29736795-h545g" event={"ID":"cb22a060-aa2f-4036-aab5-bd59453d59b8","Type":"ContainerStarted","Data":"80efc2ad9c97e36ba3c67a4386197be317f00329c15fe02788b6624c6a33cbc8"} Jul 16 13:15:01.292328 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:01.292311 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-41581b25-29736795-czpgh" event={"ID":"2fcb1d30-c0d8-4979-a128-336a8b88c32b","Type":"ContainerStarted","Data":"59e10da43374ed02ecbb0b6f9fa8cfe61750b5f9969e6ffb4ce94922ade0457a"} Jul 16 13:15:04.324700 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:04.324626 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-213b02-29736795-gzxrq" event={"ID":"692cd48f-bbaf-47d8-be8c-6ec3b690fe31","Type":"ContainerStarted","Data":"779818cfef6e55b6ea61db7ce7b8294253883d63f76b29924538d8adf93871ab"} Jul 16 13:15:04.329457 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:04.329430 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ae18e9-29736795-rqqwf" event={"ID":"846fe269-bf8f-4b63-89ed-b44fa0fbb2c1","Type":"ContainerStarted","Data":"e808f02212e1a73e2217110d5180e5a83c3a3a88b849ed3bd13f9168e13facfa"} Jul 16 13:15:04.332022 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:04.331999 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-60a47609-29736795-h545g" event={"ID":"cb22a060-aa2f-4036-aab5-bd59453d59b8","Type":"ContainerStarted","Data":"6e8869deb096278456b8c5af295875b4156ebf37b092aaf17f48278701e7c659"} Jul 16 13:15:04.334381 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:04.334301 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-a22377-29736795-pwv2d" event={"ID":"2d256401-fd4a-410b-8330-4ed55cc4054d","Type":"ContainerStarted","Data":"e96cff6f87aa1d3b9008648f7550961edc28ea6dc92310c12375d099c3b1a895"} Jul 16 13:15:04.350357 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:04.350313 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-213b02-29736795-gzxrq" podStartSLOduration=1.387823837 podStartE2EDuration="4.350298295s" totalImagesPullingTime="2.962474458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 13:15:04.348087806 +0000 UTC m=+2157.581995657" watchObservedRunningTime="2026-07-16 13:15:04.350298295 +0000 UTC m=+2157.584206144" Jul 16 13:15:04.365745 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:04.365702 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ae18e9-29736795-rqqwf" podStartSLOduration=1.2424140750000001 podStartE2EDuration="4.365690429s" totalImagesPullingTime="3.123276354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 13:15:04.363944417 +0000 UTC m=+2157.597852267" watchObservedRunningTime="2026-07-16 13:15:04.365690429 +0000 UTC m=+2157.599598328" Jul 16 13:15:04.380954 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:04.380913 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-a22377-29736795-pwv2d" podStartSLOduration=1.167431574 podStartE2EDuration="4.380900943s" totalImagesPullingTime="3.213469369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 13:15:04.380602249 +0000 UTC m=+2157.614510094" watchObservedRunningTime="2026-07-16 13:15:04.380900943 +0000 UTC m=+2157.614808784" Jul 16 13:15:04.399477 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:04.399436 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-60a47609-29736795-h545g" podStartSLOduration=1.494856794 podStartE2EDuration="4.399422593s" totalImagesPullingTime="2.904565799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 13:15:04.397521052 +0000 UTC m=+2157.631428898" watchObservedRunningTime="2026-07-16 13:15:04.399422593 +0000 UTC m=+2157.633330443" Jul 16 13:15:05.342445 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.342418 2567 generic.go:356] "Generic (PLEG): container finished" podID="2d256401-fd4a-410b-8330-4ed55cc4054d" containerID="e96cff6f87aa1d3b9008648f7550961edc28ea6dc92310c12375d099c3b1a895" exitCode=0 Jul 16 13:15:05.342806 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.342495 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-a22377-29736795-pwv2d" event={"ID":"2d256401-fd4a-410b-8330-4ed55cc4054d","Type":"ContainerDied","Data":"e96cff6f87aa1d3b9008648f7550961edc28ea6dc92310c12375d099c3b1a895"} Jul 16 13:15:05.343939 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.343913 2567 generic.go:356] "Generic (PLEG): container finished" podID="bf57401d-ce7c-4de5-a254-3cf5f8061cf6" containerID="ecfbc13be5cea7aa6cc79f736746d27b382c26ad2cc5efef99fd5b24f7fcc8e3" exitCode=0 Jul 16 13:15:05.344031 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.343984 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5a470a1b-29736795-w4kff" event={"ID":"bf57401d-ce7c-4de5-a254-3cf5f8061cf6","Type":"ContainerDied","Data":"ecfbc13be5cea7aa6cc79f736746d27b382c26ad2cc5efef99fd5b24f7fcc8e3"} Jul 16 13:15:05.345407 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.345387 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-b0d8b7c2-29736795-kr82m" event={"ID":"f9002ea6-c837-4845-b0a7-622e5073c21a","Type":"ContainerStarted","Data":"150e1bac972d9334118323485d12cfa772ce73c773bd20864cc70c76bd6bd8ba"} Jul 16 13:15:05.346786 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.346763 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-cb4eee-29736795-4d6nf" event={"ID":"6e8aad7b-5765-4b99-9e0f-ac6332d17250","Type":"ContainerStarted","Data":"3369c625d6d8cd7eec16fcdc6fd829e9f66f989ff1026f65ebe62233e0fda97f"} Jul 16 13:15:05.348147 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.348126 2567 generic.go:356] "Generic (PLEG): container finished" podID="c9d1d95e-a3bc-4550-860f-f496b4ece340" containerID="8496d694b2380227f447dedc090e767da35675da89f74255f8bfa67b5394f66d" exitCode=0 Jul 16 13:15:05.348241 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.348188 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b12c92-29736795-swrqg" event={"ID":"c9d1d95e-a3bc-4550-860f-f496b4ece340","Type":"ContainerDied","Data":"8496d694b2380227f447dedc090e767da35675da89f74255f8bfa67b5394f66d"} Jul 16 13:15:05.349425 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.349407 2567 generic.go:356] "Generic (PLEG): container finished" podID="692cd48f-bbaf-47d8-be8c-6ec3b690fe31" containerID="779818cfef6e55b6ea61db7ce7b8294253883d63f76b29924538d8adf93871ab" exitCode=0 Jul 16 13:15:05.349497 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.349463 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-213b02-29736795-gzxrq" event={"ID":"692cd48f-bbaf-47d8-be8c-6ec3b690fe31","Type":"ContainerDied","Data":"779818cfef6e55b6ea61db7ce7b8294253883d63f76b29924538d8adf93871ab"} Jul 16 13:15:05.350777 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.350755 2567 generic.go:356] "Generic (PLEG): container finished" podID="846fe269-bf8f-4b63-89ed-b44fa0fbb2c1" containerID="e808f02212e1a73e2217110d5180e5a83c3a3a88b849ed3bd13f9168e13facfa" exitCode=0 Jul 16 13:15:05.350846 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.350817 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ae18e9-29736795-rqqwf" event={"ID":"846fe269-bf8f-4b63-89ed-b44fa0fbb2c1","Type":"ContainerDied","Data":"e808f02212e1a73e2217110d5180e5a83c3a3a88b849ed3bd13f9168e13facfa"} Jul 16 13:15:05.352105 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.352085 2567 generic.go:356] "Generic (PLEG): container finished" podID="cb22a060-aa2f-4036-aab5-bd59453d59b8" containerID="6e8869deb096278456b8c5af295875b4156ebf37b092aaf17f48278701e7c659" exitCode=0 Jul 16 13:15:05.352180 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.352137 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-60a47609-29736795-h545g" event={"ID":"cb22a060-aa2f-4036-aab5-bd59453d59b8","Type":"ContainerDied","Data":"6e8869deb096278456b8c5af295875b4156ebf37b092aaf17f48278701e7c659"} Jul 16 13:15:05.353436 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.353419 2567 generic.go:356] "Generic (PLEG): container finished" podID="2fcb1d30-c0d8-4979-a128-336a8b88c32b" containerID="4ca87f1f80e5bc4fb700c6099abd4b49ef4023e984c8b8b19348b15122e80db2" exitCode=0 Jul 16 13:15:05.353500 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.353474 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-41581b25-29736795-czpgh" event={"ID":"2fcb1d30-c0d8-4979-a128-336a8b88c32b","Type":"ContainerDied","Data":"4ca87f1f80e5bc4fb700c6099abd4b49ef4023e984c8b8b19348b15122e80db2"} Jul 16 13:15:05.393162 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.393126 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-b0d8b7c2-29736795-kr82m" podStartSLOduration=2.149421908 podStartE2EDuration="5.393113158s" totalImagesPullingTime="3.24369125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 13:15:05.391683303 +0000 UTC m=+2158.625591153" watchObservedRunningTime="2026-07-16 13:15:05.393113158 +0000 UTC m=+2158.627021004" Jul 16 13:15:05.457756 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:05.457715 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-cb4eee-29736795-4d6nf" podStartSLOduration=1.891317984 podStartE2EDuration="5.457699207s" totalImagesPullingTime="3.566381223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 13:15:05.45658847 +0000 UTC m=+2158.690496320" watchObservedRunningTime="2026-07-16 13:15:05.457699207 +0000 UTC m=+2158.691607100" Jul 16 13:15:06.360558 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:06.360524 2567 generic.go:356] "Generic (PLEG): container finished" podID="6e8aad7b-5765-4b99-9e0f-ac6332d17250" containerID="3369c625d6d8cd7eec16fcdc6fd829e9f66f989ff1026f65ebe62233e0fda97f" exitCode=0 Jul 16 13:15:06.360916 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:06.360604 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-cb4eee-29736795-4d6nf" event={"ID":"6e8aad7b-5765-4b99-9e0f-ac6332d17250","Type":"ContainerDied","Data":"3369c625d6d8cd7eec16fcdc6fd829e9f66f989ff1026f65ebe62233e0fda97f"} Jul 16 13:15:06.362013 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:06.361992 2567 generic.go:356] "Generic (PLEG): container finished" podID="f9002ea6-c837-4845-b0a7-622e5073c21a" containerID="150e1bac972d9334118323485d12cfa772ce73c773bd20864cc70c76bd6bd8ba" exitCode=0 Jul 16 13:15:06.362152 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:06.362061 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-b0d8b7c2-29736795-kr82m" event={"ID":"f9002ea6-c837-4845-b0a7-622e5073c21a","Type":"ContainerDied","Data":"150e1bac972d9334118323485d12cfa772ce73c773bd20864cc70c76bd6bd8ba"} Jul 16 13:15:06.528947 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:06.528924 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-a22377-29736795-pwv2d" Jul 16 13:15:06.594776 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:06.594493 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ae18e9-29736795-rqqwf" Jul 16 13:15:06.639955 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:06.639923 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-213b02-29736795-gzxrq" Jul 16 13:15:06.650090 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:06.650008 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5a470a1b-29736795-w4kff" Jul 16 13:15:06.656591 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:06.656570 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b12c92-29736795-swrqg" Jul 16 13:15:06.659413 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:06.659135 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-60a47609-29736795-h545g" Jul 16 13:15:06.662586 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:06.662529 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-41581b25-29736795-czpgh" Jul 16 13:15:07.368578 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.368540 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b12c92-29736795-swrqg" event={"ID":"c9d1d95e-a3bc-4550-860f-f496b4ece340","Type":"ContainerDied","Data":"c9e5d4856a4c0bcafccf968e2135d6ff8ef4e8e64b68d199895500b534e9d9fb"} Jul 16 13:15:07.369002 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.368551 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-b12c92-29736795-swrqg" Jul 16 13:15:07.369002 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.368630 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9e5d4856a4c0bcafccf968e2135d6ff8ef4e8e64b68d199895500b534e9d9fb" Jul 16 13:15:07.369816 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.369799 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-213b02-29736795-gzxrq" Jul 16 13:15:07.369900 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.369795 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-213b02-29736795-gzxrq" event={"ID":"692cd48f-bbaf-47d8-be8c-6ec3b690fe31","Type":"ContainerDied","Data":"561d06b23d795b17245566c28a497cd02b14855c7db8e916b9b1fd31fae45211"} Jul 16 13:15:07.369942 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.369913 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="561d06b23d795b17245566c28a497cd02b14855c7db8e916b9b1fd31fae45211" Jul 16 13:15:07.371057 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.371032 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ae18e9-29736795-rqqwf" Jul 16 13:15:07.371166 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.371095 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ae18e9-29736795-rqqwf" event={"ID":"846fe269-bf8f-4b63-89ed-b44fa0fbb2c1","Type":"ContainerDied","Data":"f40211c12f504d054ccbc3f0e1d2bf80a8f101e2d885092046d5e71cc7bd7563"} Jul 16 13:15:07.371166 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.371142 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f40211c12f504d054ccbc3f0e1d2bf80a8f101e2d885092046d5e71cc7bd7563" Jul 16 13:15:07.372545 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.372517 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-60a47609-29736795-h545g" event={"ID":"cb22a060-aa2f-4036-aab5-bd59453d59b8","Type":"ContainerDied","Data":"80efc2ad9c97e36ba3c67a4386197be317f00329c15fe02788b6624c6a33cbc8"} Jul 16 13:15:07.372642 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.372561 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80efc2ad9c97e36ba3c67a4386197be317f00329c15fe02788b6624c6a33cbc8" Jul 16 13:15:07.372642 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.372524 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-60a47609-29736795-h545g" Jul 16 13:15:07.373925 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.373907 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-41581b25-29736795-czpgh" Jul 16 13:15:07.374034 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.373941 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-41581b25-29736795-czpgh" event={"ID":"2fcb1d30-c0d8-4979-a128-336a8b88c32b","Type":"ContainerDied","Data":"59e10da43374ed02ecbb0b6f9fa8cfe61750b5f9969e6ffb4ce94922ade0457a"} Jul 16 13:15:07.374034 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.373990 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59e10da43374ed02ecbb0b6f9fa8cfe61750b5f9969e6ffb4ce94922ade0457a" Jul 16 13:15:07.377310 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.377278 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-a22377-29736795-pwv2d" Jul 16 13:15:07.377310 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.377284 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-a22377-29736795-pwv2d" event={"ID":"2d256401-fd4a-410b-8330-4ed55cc4054d","Type":"ContainerDied","Data":"80f81d3f319d1061e046600d85f5a5f8ceef9dbf9335c963fadfb929f6e715bd"} Jul 16 13:15:07.377465 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.377320 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80f81d3f319d1061e046600d85f5a5f8ceef9dbf9335c963fadfb929f6e715bd" Jul 16 13:15:07.378958 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.378858 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5a470a1b-29736795-w4kff" Jul 16 13:15:07.378958 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.378895 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-5a470a1b-29736795-w4kff" event={"ID":"bf57401d-ce7c-4de5-a254-3cf5f8061cf6","Type":"ContainerDied","Data":"eb7ebad96f3ad0153e20026d9c7c1779a84e2cf211171c69cf2642bfc4a212b1"} Jul 16 13:15:07.378958 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.378933 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb7ebad96f3ad0153e20026d9c7c1779a84e2cf211171c69cf2642bfc4a212b1" Jul 16 13:15:07.534511 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.534488 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-b0d8b7c2-29736795-kr82m" Jul 16 13:15:07.550145 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:07.550125 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-cb4eee-29736795-4d6nf" Jul 16 13:15:08.385199 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:08.385175 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-b0d8b7c2-29736795-kr82m" Jul 16 13:15:08.385644 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:08.385174 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-b0d8b7c2-29736795-kr82m" event={"ID":"f9002ea6-c837-4845-b0a7-622e5073c21a","Type":"ContainerDied","Data":"d53f221e6d6ddd8d9d4b3259f2b09a55d21866f685fe45e95d48a3f003c0dece"} Jul 16 13:15:08.385644 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:08.385293 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d53f221e6d6ddd8d9d4b3259f2b09a55d21866f685fe45e95d48a3f003c0dece" Jul 16 13:15:08.386496 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:08.386475 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-cb4eee-29736795-4d6nf" Jul 16 13:15:08.386601 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:08.386505 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-cb4eee-29736795-4d6nf" event={"ID":"6e8aad7b-5765-4b99-9e0f-ac6332d17250","Type":"ContainerDied","Data":"b6c032aa2cb0d1107a6726f49c09498eb4481c32ecb7c9f817d4c5b8b828df5e"} Jul 16 13:15:08.386601 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:08.386540 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b6c032aa2cb0d1107a6726f49c09498eb4481c32ecb7c9f817d4c5b8b828df5e" Jul 16 13:15:09.085140 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:09.085112 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:15:48.577718 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:15:48.577682 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:16:06.381359 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:06.381326 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:16:31.393664 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:31.393581 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:16:33.268867 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.268832 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b"] Jul 16 13:16:33.269487 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269428 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e8aad7b-5765-4b99-9e0f-ac6332d17250" containerName="cleanup" Jul 16 13:16:33.269487 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269443 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e8aad7b-5765-4b99-9e0f-ac6332d17250" containerName="cleanup" Jul 16 13:16:33.269487 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269454 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9002ea6-c837-4845-b0a7-622e5073c21a" containerName="cleanup" Jul 16 13:16:33.269487 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269462 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9002ea6-c837-4845-b0a7-622e5073c21a" containerName="cleanup" Jul 16 13:16:33.269487 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269474 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf57401d-ce7c-4de5-a254-3cf5f8061cf6" containerName="cleanup" Jul 16 13:16:33.269487 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269481 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf57401d-ce7c-4de5-a254-3cf5f8061cf6" containerName="cleanup" Jul 16 13:16:33.269487 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269491 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="846fe269-bf8f-4b63-89ed-b44fa0fbb2c1" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269496 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="846fe269-bf8f-4b63-89ed-b44fa0fbb2c1" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269502 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d256401-fd4a-410b-8330-4ed55cc4054d" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269508 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d256401-fd4a-410b-8330-4ed55cc4054d" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269522 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fcb1d30-c0d8-4979-a128-336a8b88c32b" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269527 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fcb1d30-c0d8-4979-a128-336a8b88c32b" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269542 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9d1d95e-a3bc-4550-860f-f496b4ece340" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269550 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9d1d95e-a3bc-4550-860f-f496b4ece340" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269558 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb22a060-aa2f-4036-aab5-bd59453d59b8" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269563 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb22a060-aa2f-4036-aab5-bd59453d59b8" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269571 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="692cd48f-bbaf-47d8-be8c-6ec3b690fe31" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269576 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="692cd48f-bbaf-47d8-be8c-6ec3b690fe31" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269642 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="692cd48f-bbaf-47d8-be8c-6ec3b690fe31" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269652 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9002ea6-c837-4845-b0a7-622e5073c21a" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269660 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9d1d95e-a3bc-4550-860f-f496b4ece340" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269667 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fcb1d30-c0d8-4979-a128-336a8b88c32b" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269674 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="846fe269-bf8f-4b63-89ed-b44fa0fbb2c1" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269679 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d256401-fd4a-410b-8330-4ed55cc4054d" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269685 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb22a060-aa2f-4036-aab5-bd59453d59b8" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269690 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e8aad7b-5765-4b99-9e0f-ac6332d17250" containerName="cleanup" Jul 16 13:16:33.269919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.269697 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf57401d-ce7c-4de5-a254-3cf5f8061cf6" containerName="cleanup" Jul 16 13:16:33.272013 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.271993 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.276023 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.276002 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b12c92\"/\"openshift-service-ca.crt\"" Jul 16 13:16:33.276023 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.276012 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b12c92\"/\"default-dockercfg-5n2sv\"" Jul 16 13:16:33.276191 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.276066 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b12c92\"/\"kube-root-ca.crt\"" Jul 16 13:16:33.276191 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.276126 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b12c92\"/\"rate-test-model-b12c92-kserve-self-signed-certs\"" Jul 16 13:16:33.283984 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.283962 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b"] Jul 16 13:16:33.456988 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.456953 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-model-cache\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.456988 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.456987 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-tmp-dir\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.457200 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.457013 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf225\" (UniqueName: \"kubernetes.io/projected/df63a97f-0658-44ab-b218-f128216a17f9-kube-api-access-pf225\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.457200 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.457149 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-home\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.457273 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.457233 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df63a97f-0658-44ab-b218-f128216a17f9-tls-certs\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.457373 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.457354 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-dshm\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.558700 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.558617 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df63a97f-0658-44ab-b218-f128216a17f9-tls-certs\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.558700 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.558696 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-dshm\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.558917 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.558730 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-model-cache\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.558917 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.558746 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-tmp-dir\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.558917 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.558762 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf225\" (UniqueName: \"kubernetes.io/projected/df63a97f-0658-44ab-b218-f128216a17f9-kube-api-access-pf225\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.558917 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.558781 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-home\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.559240 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.559218 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-model-cache\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.559328 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.559262 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-home\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.559432 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.559416 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-tmp-dir\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.561323 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.561296 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-dshm\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.561545 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.561528 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df63a97f-0658-44ab-b218-f128216a17f9-tls-certs\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.586878 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.586847 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf225\" (UniqueName: \"kubernetes.io/projected/df63a97f-0658-44ab-b218-f128216a17f9-kube-api-access-pf225\") pod \"rate-test-model-b12c92-kserve-fb794f9dc-fjv4b\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:33.881753 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:33.881665 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:34.022729 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:34.022706 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b"] Jul 16 13:16:34.025095 ip-10-0-140-165 kubenswrapper[2567]: W0716 13:16:34.025029 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf63a97f_0658_44ab_b218_f128216a17f9.slice/crio-0c150659370cd7fa7c6bbe8d3c92034d09349eba7c7e98a937f60da08496cb80 WatchSource:0}: Error finding container 0c150659370cd7fa7c6bbe8d3c92034d09349eba7c7e98a937f60da08496cb80: Status 404 returned error can't find the container with id 0c150659370cd7fa7c6bbe8d3c92034d09349eba7c7e98a937f60da08496cb80 Jul 16 13:16:34.856748 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:34.856710 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" event={"ID":"df63a97f-0658-44ab-b218-f128216a17f9","Type":"ContainerStarted","Data":"075418b0caadf895ec062e0e0c0599b146daa7a9fbb865562aaebb9150ec4436"} Jul 16 13:16:34.857177 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:34.856762 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" event={"ID":"df63a97f-0658-44ab-b218-f128216a17f9","Type":"ContainerStarted","Data":"0c150659370cd7fa7c6bbe8d3c92034d09349eba7c7e98a937f60da08496cb80"} Jul 16 13:16:34.877752 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:34.877700 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" podStartSLOduration=1.877684277 podStartE2EDuration="1.877684277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 13:16:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 13:16:34.8756258 +0000 UTC m=+2248.109533650" watchObservedRunningTime="2026-07-16 13:16:34.877684277 +0000 UTC m=+2248.111592225" Jul 16 13:16:43.882589 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:43.882548 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:43.882971 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:43.882606 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:43.895012 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:43.894981 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:43.914464 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:43.914431 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:16:46.287492 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:16:46.287458 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:17:14.086106 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:17:14.086076 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:17:31.582759 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:17:31.582715 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:17:53.491656 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:17:53.491624 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:17:57.716780 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:17:57.716747 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b"] Jul 16 13:18:11.680872 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:11.680841 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:18:12.737362 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:12.737305 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" podUID="df63a97f-0658-44ab-b218-f128216a17f9" containerName="main" containerID="cri-o://075418b0caadf895ec062e0e0c0599b146daa7a9fbb865562aaebb9150ec4436" gracePeriod=45 Jul 16 13:18:13.006256 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.006226 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:18:13.052925 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.052900 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-dshm\") pod \"df63a97f-0658-44ab-b218-f128216a17f9\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " Jul 16 13:18:13.053067 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.052943 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df63a97f-0658-44ab-b218-f128216a17f9-tls-certs\") pod \"df63a97f-0658-44ab-b218-f128216a17f9\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " Jul 16 13:18:13.053067 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.053005 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-model-cache\") pod \"df63a97f-0658-44ab-b218-f128216a17f9\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " Jul 16 13:18:13.053155 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.053076 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf225\" (UniqueName: \"kubernetes.io/projected/df63a97f-0658-44ab-b218-f128216a17f9-kube-api-access-pf225\") pod \"df63a97f-0658-44ab-b218-f128216a17f9\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " Jul 16 13:18:13.053155 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.053101 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-tmp-dir\") pod \"df63a97f-0658-44ab-b218-f128216a17f9\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " Jul 16 13:18:13.053155 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.053118 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-home\") pod \"df63a97f-0658-44ab-b218-f128216a17f9\" (UID: \"df63a97f-0658-44ab-b218-f128216a17f9\") " Jul 16 13:18:13.053441 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.053374 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "df63a97f-0658-44ab-b218-f128216a17f9" (UID: "df63a97f-0658-44ab-b218-f128216a17f9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 13:18:13.053441 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.053389 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-model-cache" (OuterVolumeSpecName: "model-cache") pod "df63a97f-0658-44ab-b218-f128216a17f9" (UID: "df63a97f-0658-44ab-b218-f128216a17f9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 13:18:13.053603 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.053518 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-home" (OuterVolumeSpecName: "home") pod "df63a97f-0658-44ab-b218-f128216a17f9" (UID: "df63a97f-0658-44ab-b218-f128216a17f9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 13:18:13.055289 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.055265 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df63a97f-0658-44ab-b218-f128216a17f9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "df63a97f-0658-44ab-b218-f128216a17f9" (UID: "df63a97f-0658-44ab-b218-f128216a17f9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 13:18:13.055367 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.055315 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df63a97f-0658-44ab-b218-f128216a17f9-kube-api-access-pf225" (OuterVolumeSpecName: "kube-api-access-pf225") pod "df63a97f-0658-44ab-b218-f128216a17f9" (UID: "df63a97f-0658-44ab-b218-f128216a17f9"). InnerVolumeSpecName "kube-api-access-pf225". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 13:18:13.055423 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.055390 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-dshm" (OuterVolumeSpecName: "dshm") pod "df63a97f-0658-44ab-b218-f128216a17f9" (UID: "df63a97f-0658-44ab-b218-f128216a17f9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 13:18:13.154232 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.154207 2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 13:18:13.154232 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.154230 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf225\" (UniqueName: \"kubernetes.io/projected/df63a97f-0658-44ab-b218-f128216a17f9-kube-api-access-pf225\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 13:18:13.154362 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.154241 2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 13:18:13.154362 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.154251 2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 13:18:13.154362 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.154258 2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df63a97f-0658-44ab-b218-f128216a17f9-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 13:18:13.154362 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.154265 2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df63a97f-0658-44ab-b218-f128216a17f9-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\"" Jul 16 13:18:13.416479 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.416396 2567 generic.go:356] "Generic (PLEG): container finished" podID="df63a97f-0658-44ab-b218-f128216a17f9" containerID="075418b0caadf895ec062e0e0c0599b146daa7a9fbb865562aaebb9150ec4436" exitCode=0 Jul 16 13:18:13.416479 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.416442 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" Jul 16 13:18:13.416670 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.416474 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" event={"ID":"df63a97f-0658-44ab-b218-f128216a17f9","Type":"ContainerDied","Data":"075418b0caadf895ec062e0e0c0599b146daa7a9fbb865562aaebb9150ec4436"} Jul 16 13:18:13.416670 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.416524 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b" event={"ID":"df63a97f-0658-44ab-b218-f128216a17f9","Type":"ContainerDied","Data":"0c150659370cd7fa7c6bbe8d3c92034d09349eba7c7e98a937f60da08496cb80"} Jul 16 13:18:13.416670 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.416545 2567 scope.go:117] "RemoveContainer" containerID="075418b0caadf895ec062e0e0c0599b146daa7a9fbb865562aaebb9150ec4436" Jul 16 13:18:13.426147 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.426130 2567 scope.go:117] "RemoveContainer" containerID="075418b0caadf895ec062e0e0c0599b146daa7a9fbb865562aaebb9150ec4436" Jul 16 13:18:13.426402 ip-10-0-140-165 kubenswrapper[2567]: E0716 13:18:13.426384 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"075418b0caadf895ec062e0e0c0599b146daa7a9fbb865562aaebb9150ec4436\": container with ID starting with 075418b0caadf895ec062e0e0c0599b146daa7a9fbb865562aaebb9150ec4436 not found: ID does not exist" containerID="075418b0caadf895ec062e0e0c0599b146daa7a9fbb865562aaebb9150ec4436" Jul 16 13:18:13.426463 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.426408 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"075418b0caadf895ec062e0e0c0599b146daa7a9fbb865562aaebb9150ec4436"} err="failed to get container status \"075418b0caadf895ec062e0e0c0599b146daa7a9fbb865562aaebb9150ec4436\": rpc error: code = NotFound desc = could not find container \"075418b0caadf895ec062e0e0c0599b146daa7a9fbb865562aaebb9150ec4436\": container with ID starting with 075418b0caadf895ec062e0e0c0599b146daa7a9fbb865562aaebb9150ec4436 not found: ID does not exist" Jul 16 13:18:13.438569 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.438544 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b"] Jul 16 13:18:13.441824 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.441804 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b12c92/rate-test-model-b12c92-kserve-fb794f9dc-fjv4b"] Jul 16 13:18:13.640365 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:13.640323 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df63a97f-0658-44ab-b218-f128216a17f9" path="/var/lib/kubelet/pods/df63a97f-0658-44ab-b218-f128216a17f9/volumes" Jul 16 13:18:28.889003 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:28.888965 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:18:39.241067 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:39.241023 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wrhvj_674f92e7-2580-41cb-bf8a-5e735af66124/manager/0.log" Jul 16 13:18:39.377323 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:39.377289 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6sqdk_1c2296a3-023d-4d98-be47-2aa0bebb1f4f/manager/0.log" Jul 16 13:18:39.387687 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:39.387661 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ltj7k"] Jul 16 13:18:39.639834 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:39.639750 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gbk4m_f28df006-ac3a-485b-860a-bbab526b2a9c/server/0.log" Jul 16 13:18:39.897374 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:39.897292 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65f8bdf75c-4dkrw_ce7229d5-45f5-49d3-90eb-5c3721cc44a8/copy-manifests/0.log" Jul 16 13:18:39.904062 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:39.904029 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65f8bdf75c-4dkrw_ce7229d5-45f5-49d3-90eb-5c3721cc44a8/manager/0.log" Jul 16 13:18:41.898377 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:41.898344 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-e2e-shared-b-8d69f8-dc9bffdb8-lcz46_b04e5aa7-a1cf-4b80-973e-0d198480a39c/maas-api/0.log" Jul 16 13:18:42.273403 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:42.273375 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-ait-60a47609-29736795-h545g_cb22a060-aa2f-4036-aab5-bd59453d59b8/cleanup/0.log" Jul 16 13:18:42.387373 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:42.387347 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-api-a-213b02-29736795-gzxrq_692cd48f-bbaf-47d8-be8c-6ec3b690fe31/cleanup/0.log" Jul 16 13:18:42.505729 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:42.505699 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-api-b-a22377-29736795-pwv2d_2d256401-fd4a-410b-8330-4ed55cc4054d/cleanup/0.log" Jul 16 13:18:42.620983 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:42.620910 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-derive-b0d8b7c2-29736795-kr82m_f9002ea6-c837-4845-b0a7-622e5073c21a/cleanup/0.log" Jul 16 13:18:42.733901 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:42.733873 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-inf-a-ae18e9-29736795-rqqwf_846fe269-bf8f-4b63-89ed-b44fa0fbb2c1/cleanup/0.log" Jul 16 13:18:42.849306 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:42.849278 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-inf-b-cb4eee-29736795-4d6nf_6e8aad7b-5765-4b99-9e0f-ac6332d17250/cleanup/0.log" Jul 16 13:18:42.960678 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:42.960649 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-migrate-5a470a1b-29736795-w4kff_bf57401d-ce7c-4de5-a254-3cf5f8061cf6/cleanup/0.log" Jul 16 13:18:43.079888 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:43.079858 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-mt-41581b25-29736795-czpgh_2fcb1d30-c0d8-4979-a128-336a8b88c32b/cleanup/0.log" Jul 16 13:18:43.198607 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:43.198578 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-shared-a-b12c92-29736795-swrqg_c9d1d95e-a3bc-4550-860f-f496b4ece340/cleanup/0.log" Jul 16 13:18:44.980243 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:44.980208 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lcp54_6140e625-a1e4-4c56-a7b4-b9ae2cff409c/manager/0.log" Jul 16 13:18:45.095063 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:45.095023 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-gzhrc_6326a8bf-e22e-420a-9981-811c3b238cb7/manager/0.log" Jul 16 13:18:45.208831 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:45.208807 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-n8zll_07e4e793-4349-48b9-b10b-9d16821b91ec/kuadrant-console-plugin/0.log" Jul 16 13:18:45.328437 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:45.328372 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xp6gr_4d4d0198-ddd2-4fbe-a81b-a9ac12923570/registry-server/0.log" Jul 16 13:18:45.459194 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:45.459167 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qbjrl_279c83b0-aa6b-40dd-8d2c-804fd31182ef/manager/0.log" Jul 16 13:18:45.585230 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:45.585164 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-ltj7k_496c8746-66a2-433a-9c39-830fb301655a/limitador/0.log" Jul 16 13:18:45.704013 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:45.703993 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4vh8c_4d997cae-661d-4489-88fe-d4db001fd5ce/manager/0.log" Jul 16 13:18:46.349669 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:46.349639 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6678559b48-jsd7l_dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0/kube-auth-proxy/0.log" Jul 16 13:18:46.607972 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:46.607889 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xq6gw_b139d0d7-91aa-4263-815e-2ed4038b0ae0/istio-proxy/1.log" Jul 16 13:18:46.981389 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:46.981361 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d87ff988b-lrrxq_6f912bf2-e08f-42ec-b4f2-92edf90961ba/router/0.log" Jul 16 13:18:47.723133 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:47.723102 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh8lz4_73e94106-9c1a-4809-bc51-cbaf135769af/main/0.log" Jul 16 13:18:47.838208 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:47.838183 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-pb986_95fd15d5-9b3a-4cc4-98df-2faf116d5bfe/main/0.log" Jul 16 13:18:47.957775 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:47.957748 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-6jt7z_47ea0e66-e55a-4612-92bd-5044973e8eb8/main/0.log" Jul 16 13:18:57.234156 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:57.234128 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-78h5z_f0a26b92-3040-40c7-904f-7c8112a1f5e5/global-pull-secret-syncer/0.log" Jul 16 13:18:57.395174 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:57.395146 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6jbsz_a91426dc-0372-43c1-9929-339429bd1d6a/konnectivity-agent/0.log" Jul 16 13:18:57.484955 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:18:57.484880 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-165.ec2.internal_acfaee3e8f27ae7b52fdbaeafbb5fcbe/haproxy/0.log" Jul 16 13:19:02.213205 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:02.213179 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lcp54_6140e625-a1e4-4c56-a7b4-b9ae2cff409c/manager/0.log" Jul 16 13:19:02.252980 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:02.252957 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-gzhrc_6326a8bf-e22e-420a-9981-811c3b238cb7/manager/0.log" Jul 16 13:19:02.280498 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:02.280474 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-n8zll_07e4e793-4349-48b9-b10b-9d16821b91ec/kuadrant-console-plugin/0.log" Jul 16 13:19:02.317307 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:02.317283 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xp6gr_4d4d0198-ddd2-4fbe-a81b-a9ac12923570/registry-server/0.log" Jul 16 13:19:02.442183 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:02.442154 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qbjrl_279c83b0-aa6b-40dd-8d2c-804fd31182ef/manager/0.log" Jul 16 13:19:02.473984 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:02.473922 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-ltj7k_496c8746-66a2-433a-9c39-830fb301655a/limitador/0.log" Jul 16 13:19:02.530753 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:02.530729 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4vh8c_4d997cae-661d-4489-88fe-d4db001fd5ce/manager/0.log" Jul 16 13:19:04.388649 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:04.388624 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-z7mdk_1c8777d3-5209-48ac-843c-35d07c0d30da/cluster-monitoring-operator/0.log" Jul 16 13:19:04.448061 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:04.448023 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-rnwsr_d97e9e1d-2485-429c-9e77-6dd6fa67d02d/kube-state-metrics/0.log" Jul 16 13:19:04.493594 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:04.493570 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-rnwsr_d97e9e1d-2485-429c-9e77-6dd6fa67d02d/kube-rbac-proxy-main/0.log" Jul 16 13:19:04.543482 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:04.543459 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-rnwsr_d97e9e1d-2485-429c-9e77-6dd6fa67d02d/kube-rbac-proxy-self/0.log" Jul 16 13:19:04.916417 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:04.916384 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s5w6f_8ad22e82-e80f-41b2-b40c-a9562db3e320/node-exporter/0.log" Jul 16 13:19:04.949371 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:04.949341 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s5w6f_8ad22e82-e80f-41b2-b40c-a9562db3e320/kube-rbac-proxy/0.log" Jul 16 13:19:04.984357 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:04.984337 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s5w6f_8ad22e82-e80f-41b2-b40c-a9562db3e320/init-textfile/0.log" Jul 16 13:19:05.471765 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:05.471738 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-vtj5k_4b2d8894-e54d-4464-81b6-0575052d69b8/prometheus-operator/0.log" Jul 16 13:19:05.511908 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:05.511882 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-vtj5k_4b2d8894-e54d-4464-81b6-0575052d69b8/kube-rbac-proxy/0.log" Jul 16 13:19:07.128293 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:07.128265 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-wpzqj_06136c04-642c-4fc9-806b-40424fd711bb/networking-console-plugin/0.log" Jul 16 13:19:07.374099 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:07.374035 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 13:19:07.376984 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:07.376965 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 13:19:07.383612 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:07.383561 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 13:19:07.386451 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:07.386434 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 13:19:07.779906 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:07.779876 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/2.log" Jul 16 13:19:07.783746 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:07.783715 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-87f8h_15e36e16-a26a-4369-830c-c774fde70bb8/console-operator/3.log" Jul 16 13:19:07.991958 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:07.991934 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5"] Jul 16 13:19:07.992328 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:07.992316 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df63a97f-0658-44ab-b218-f128216a17f9" containerName="main" Jul 16 13:19:07.992390 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:07.992330 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="df63a97f-0658-44ab-b218-f128216a17f9" containerName="main" Jul 16 13:19:07.992428 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:07.992414 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="df63a97f-0658-44ab-b218-f128216a17f9" containerName="main" Jul 16 13:19:07.995590 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:07.995572 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:07.998349 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:07.998326 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-48rx4\"/\"kube-root-ca.crt\"" Jul 16 13:19:07.998446 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:07.998359 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-48rx4\"/\"default-dockercfg-rvmwq\"" Jul 16 13:19:07.998446 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:07.998362 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-48rx4\"/\"openshift-service-ca.crt\"" Jul 16 13:19:08.014474 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.014453 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5"] Jul 16 13:19:08.110910 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.110851 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6b43af71-221c-4139-b4a5-a94a1a3b9e40-proc\") pod \"perf-node-gather-daemonset-8htv5\" (UID: \"6b43af71-221c-4139-b4a5-a94a1a3b9e40\") " pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.111011 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.110917 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6b43af71-221c-4139-b4a5-a94a1a3b9e40-podres\") pod \"perf-node-gather-daemonset-8htv5\" (UID: \"6b43af71-221c-4139-b4a5-a94a1a3b9e40\") " pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.111011 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.110974 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b43af71-221c-4139-b4a5-a94a1a3b9e40-lib-modules\") pod \"perf-node-gather-daemonset-8htv5\" (UID: \"6b43af71-221c-4139-b4a5-a94a1a3b9e40\") " pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.111110 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.111009 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp97c\" (UniqueName: \"kubernetes.io/projected/6b43af71-221c-4139-b4a5-a94a1a3b9e40-kube-api-access-xp97c\") pod \"perf-node-gather-daemonset-8htv5\" (UID: \"6b43af71-221c-4139-b4a5-a94a1a3b9e40\") " pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.111110 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.111058 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b43af71-221c-4139-b4a5-a94a1a3b9e40-sys\") pod \"perf-node-gather-daemonset-8htv5\" (UID: \"6b43af71-221c-4139-b4a5-a94a1a3b9e40\") " pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.211567 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.211531 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6b43af71-221c-4139-b4a5-a94a1a3b9e40-podres\") pod \"perf-node-gather-daemonset-8htv5\" (UID: \"6b43af71-221c-4139-b4a5-a94a1a3b9e40\") " pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.211919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.211576 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b43af71-221c-4139-b4a5-a94a1a3b9e40-lib-modules\") pod \"perf-node-gather-daemonset-8htv5\" (UID: \"6b43af71-221c-4139-b4a5-a94a1a3b9e40\") " pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.211919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.211688 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6b43af71-221c-4139-b4a5-a94a1a3b9e40-podres\") pod \"perf-node-gather-daemonset-8htv5\" (UID: \"6b43af71-221c-4139-b4a5-a94a1a3b9e40\") " pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.211919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.211702 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b43af71-221c-4139-b4a5-a94a1a3b9e40-lib-modules\") pod \"perf-node-gather-daemonset-8htv5\" (UID: \"6b43af71-221c-4139-b4a5-a94a1a3b9e40\") " pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.211919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.211691 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp97c\" (UniqueName: \"kubernetes.io/projected/6b43af71-221c-4139-b4a5-a94a1a3b9e40-kube-api-access-xp97c\") pod \"perf-node-gather-daemonset-8htv5\" (UID: \"6b43af71-221c-4139-b4a5-a94a1a3b9e40\") " pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.211919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.211768 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b43af71-221c-4139-b4a5-a94a1a3b9e40-sys\") pod \"perf-node-gather-daemonset-8htv5\" (UID: \"6b43af71-221c-4139-b4a5-a94a1a3b9e40\") " pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.211919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.211807 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6b43af71-221c-4139-b4a5-a94a1a3b9e40-proc\") pod \"perf-node-gather-daemonset-8htv5\" (UID: \"6b43af71-221c-4139-b4a5-a94a1a3b9e40\") " pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.211919 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.211867 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b43af71-221c-4139-b4a5-a94a1a3b9e40-sys\") pod \"perf-node-gather-daemonset-8htv5\" (UID: \"6b43af71-221c-4139-b4a5-a94a1a3b9e40\") " pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.212205 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.211948 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6b43af71-221c-4139-b4a5-a94a1a3b9e40-proc\") pod \"perf-node-gather-daemonset-8htv5\" (UID: \"6b43af71-221c-4139-b4a5-a94a1a3b9e40\") " pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.219969 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.219951 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp97c\" (UniqueName: \"kubernetes.io/projected/6b43af71-221c-4139-b4a5-a94a1a3b9e40-kube-api-access-xp97c\") pod \"perf-node-gather-daemonset-8htv5\" (UID: \"6b43af71-221c-4139-b4a5-a94a1a3b9e40\") " pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.286510 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.286483 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-ccc56667d-725m6_7254a9a0-5304-415c-86a5-f28c720ed7a4/console/0.log" Jul 16 13:19:08.305765 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.305740 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.438318 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.438294 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5"] Jul 16 13:19:08.440694 ip-10-0-140-165 kubenswrapper[2567]: W0716 13:19:08.440661 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6b43af71_221c_4139_b4a5_a94a1a3b9e40.slice/crio-fe86f84bd19ef9463abe8f5e29438ae2c37f65498e6de4bf8c8369861f9cb608 WatchSource:0}: Error finding container fe86f84bd19ef9463abe8f5e29438ae2c37f65498e6de4bf8c8369861f9cb608: Status 404 returned error can't find the container with id fe86f84bd19ef9463abe8f5e29438ae2c37f65498e6de4bf8c8369861f9cb608 Jul 16 13:19:08.710841 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.710807 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" event={"ID":"6b43af71-221c-4139-b4a5-a94a1a3b9e40","Type":"ContainerStarted","Data":"32abc6d5fc2c9b6dc17772154ce41f681a28b913fcd6c480db47c7c23f70e989"} Jul 16 13:19:08.710965 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.710859 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" event={"ID":"6b43af71-221c-4139-b4a5-a94a1a3b9e40","Type":"ContainerStarted","Data":"fe86f84bd19ef9463abe8f5e29438ae2c37f65498e6de4bf8c8369861f9cb608"} Jul 16 13:19:08.710965 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.710895 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:08.734072 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:08.732170 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" podStartSLOduration=1.73215132 podStartE2EDuration="1.73215132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 13:19:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 13:19:08.728455022 +0000 UTC m=+2401.962362876" watchObservedRunningTime="2026-07-16 13:19:08.73215132 +0000 UTC m=+2401.966059170" Jul 16 13:19:09.921001 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:09.920972 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-67vll_f0f292a3-1e1b-49b2-a900-14563f2a9e15/dns/0.log" Jul 16 13:19:09.942281 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:09.942253 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-67vll_f0f292a3-1e1b-49b2-a900-14563f2a9e15/kube-rbac-proxy/0.log" Jul 16 13:19:10.078570 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:10.078549 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-krt5w_a88360e6-cc58-4620-95c5-ce491044458c/dns-node-resolver/0.log" Jul 16 13:19:10.626638 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:10.626606 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-76d4b7d8fd-2b8r7_88164a6d-4e64-4b9e-9805-400ded0814f1/registry/0.log" Jul 16 13:19:10.647619 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:10.647595 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-42h29_c9ad3582-7d79-470d-a2e0-cfcb46eef262/node-ca/0.log" Jul 16 13:19:11.860263 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:11.860239 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6678559b48-jsd7l_dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0/kube-auth-proxy/0.log" Jul 16 13:19:11.908394 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:11.908363 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xq6gw_b139d0d7-91aa-4263-815e-2ed4038b0ae0/istio-proxy/0.log" Jul 16 13:19:11.935010 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:11.934990 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xq6gw_b139d0d7-91aa-4263-815e-2ed4038b0ae0/istio-proxy/1.log" Jul 16 13:19:12.032066 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:12.032024 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d87ff988b-lrrxq_6f912bf2-e08f-42ec-b4f2-92edf90961ba/router/0.log" Jul 16 13:19:12.713414 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:12.713388 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j8t4c_3e2c9d30-2f89-4a05-984d-08424b633851/serve-healthcheck-canary/0.log" Jul 16 13:19:13.216209 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:13.216182 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-88nwz_bda0009f-40f8-4fa0-9c96-ef016877e844/kube-rbac-proxy/0.log" Jul 16 13:19:13.236251 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:13.236233 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-88nwz_bda0009f-40f8-4fa0-9c96-ef016877e844/exporter/0.log" Jul 16 13:19:13.258107 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:13.258089 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-88nwz_bda0009f-40f8-4fa0-9c96-ef016877e844/extractor/0.log" Jul 16 13:19:14.725146 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:14.725122 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-48rx4/perf-node-gather-daemonset-8htv5" Jul 16 13:19:15.511228 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:15.511197 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wrhvj_674f92e7-2580-41cb-bf8a-5e735af66124/manager/0.log" Jul 16 13:19:15.611644 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:15.611616 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6sqdk_1c2296a3-023d-4d98-be47-2aa0bebb1f4f/manager/0.log" Jul 16 13:19:15.854194 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:15.854117 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gbk4m_f28df006-ac3a-485b-860a-bbab526b2a9c/server/0.log" Jul 16 13:19:15.987622 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:15.987592 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65f8bdf75c-4dkrw_ce7229d5-45f5-49d3-90eb-5c3721cc44a8/manager/0.log" Jul 16 13:19:16.010687 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:16.010667 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65f8bdf75c-4dkrw_ce7229d5-45f5-49d3-90eb-5c3721cc44a8/copy-manifests/0.log" Jul 16 13:19:17.545713 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:17.545684 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6678559b48-jsd7l_dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0/kube-auth-proxy/0.log" Jul 16 13:19:17.586763 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:17.586736 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xq6gw_b139d0d7-91aa-4263-815e-2ed4038b0ae0/istio-proxy/0.log" Jul 16 13:19:17.613197 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:17.613173 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xq6gw_b139d0d7-91aa-4263-815e-2ed4038b0ae0/istio-proxy/1.log" Jul 16 13:19:17.700994 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:17.700968 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d87ff988b-lrrxq_6f912bf2-e08f-42ec-b4f2-92edf90961ba/router/0.log" Jul 16 13:19:19.258720 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:19.258643 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6678559b48-jsd7l_dddc34cc-51e4-4a15-b5a3-2b2b4cb422c0/kube-auth-proxy/0.log" Jul 16 13:19:19.292616 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:19.292590 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xq6gw_b139d0d7-91aa-4263-815e-2ed4038b0ae0/istio-proxy/0.log" Jul 16 13:19:19.319552 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:19.319529 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xq6gw_b139d0d7-91aa-4263-815e-2ed4038b0ae0/istio-proxy/1.log" Jul 16 13:19:19.398062 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:19.398023 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d87ff988b-lrrxq_6f912bf2-e08f-42ec-b4f2-92edf90961ba/router/0.log" Jul 16 13:19:24.150837 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:24.150804 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v4lc5_5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9/kube-multus-additional-cni-plugins/0.log" Jul 16 13:19:24.170364 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:24.170345 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v4lc5_5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9/egress-router-binary-copy/0.log" Jul 16 13:19:24.192181 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:24.192162 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v4lc5_5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9/cni-plugins/0.log" Jul 16 13:19:24.212721 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:24.212701 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v4lc5_5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9/bond-cni-plugin/0.log" Jul 16 13:19:24.233084 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:24.233066 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v4lc5_5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9/routeoverride-cni/0.log" Jul 16 13:19:24.252040 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:24.252022 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v4lc5_5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9/whereabouts-cni-bincopy/0.log" Jul 16 13:19:24.272578 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:24.272562 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v4lc5_5d1ebc5e-2b7a-4e52-a17b-99c9c66ff5a9/whereabouts-cni/0.log" Jul 16 13:19:24.498745 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:24.498712 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lph7m_ae642324-8238-4122-8aa2-055642caafdd/kube-multus/0.log" Jul 16 13:19:24.518083 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:24.518063 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-d6kvg_8892e9d1-ce2b-47d1-af07-27d1e94dd31c/network-metrics-daemon/0.log" Jul 16 13:19:24.537369 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:24.537352 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-d6kvg_8892e9d1-ce2b-47d1-af07-27d1e94dd31c/kube-rbac-proxy/0.log" Jul 16 13:19:25.674373 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:25.674346 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-controller/0.log" Jul 16 13:19:25.690427 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:25.690403 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/1.log" Jul 16 13:19:25.690641 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:25.690626 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovn-acl-logging/0.log" Jul 16 13:19:25.708354 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:25.708330 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/kube-rbac-proxy-node/0.log" Jul 16 13:19:25.727017 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:25.726997 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/kube-rbac-proxy-ovn-metrics/0.log" Jul 16 13:19:25.746660 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:25.746641 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/northd/0.log" Jul 16 13:19:25.769230 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:25.769214 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/nbdb/0.log" Jul 16 13:19:25.789259 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:25.789240 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/sbdb/0.log" Jul 16 13:19:25.838059 ip-10-0-140-165 kubenswrapper[2567]: I0716 13:19:25.838023 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s729s_aa1afabc-1ae8-4482-b6b4-b6ae5cdf88d9/ovnkube-controller/0.log"