Jul 15 16:56:23.832567 ip-10-0-134-118 systemd[1]: Starting Kubernetes Kubelet... Jul 15 16:56:24.342358 ip-10-0-134-118 systemd[1]: Started Kubernetes Kubelet. Jul 15 16:56:24.377356 ip-10-0-134-118 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 16:56:24.377356 ip-10-0-134-118 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 16:56:24.377356 ip-10-0-134-118 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 16:56:24.377356 ip-10-0-134-118 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 16:56:24.377356 ip-10-0-134-118 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 16:56:24.377356 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.267243 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 16:56:24.377356 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271402 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:56:24.377356 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271413 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:56:24.377356 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271417 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:56:24.377356 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271420 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:56:24.377356 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271423 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:56:24.377356 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271427 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:56:24.377356 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271431 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:56:24.377356 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271433 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271436 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271439 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271441 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271444 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271446 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271449 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271452 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271454 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271456 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271459 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271462 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271465 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271467 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271470 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271473 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271475 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271478 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271481 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271483 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:56:24.378330 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271486 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271489 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271491 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271494 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271497 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271499 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271502 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271504 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271507 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271510 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271512 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271516 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271519 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271521 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271524 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271526 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271529 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271532 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271535 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271540 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:56:24.379135 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271544 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271546 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271549 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271552 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271555 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271557 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271560 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271563 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271565 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271568 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271570 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271572 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271575 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271579 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271581 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271584 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271586 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271589 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271592 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271594 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:56:24.380208 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271597 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271601 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271604 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271607 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271609 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271612 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271614 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271617 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271620 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271622 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271625 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271627 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271631 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271634 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271638 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271640 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271643 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271646 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271649 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:56:24.381152 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271651 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271654 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271656 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271659 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271663 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271666 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271669 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271672 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271674 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271676 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271679 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271682 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.271684 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272377 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272384 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272387 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272390 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272393 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272396 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272398 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:56:24.381968 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272401 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272405 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272407 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272410 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272412 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272414 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272417 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272420 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272422 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272425 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272428 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272431 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272433 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272437 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272439 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272442 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272447 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272450 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272452 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272455 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:56:24.382640 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272457 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272460 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272462 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272465 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272467 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272470 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272472 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272475 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272477 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272480 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272482 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272485 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272488 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272491 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272493 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272496 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272498 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272501 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272503 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:56:24.383222 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272506 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272508 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272511 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272513 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272517 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272522 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272524 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272528 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272531 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272536 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272538 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272541 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272543 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272546 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272549 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272551 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272554 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272556 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272559 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:56:24.383800 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272562 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272565 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272567 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272570 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272572 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272575 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272578 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272582 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272585 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272587 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272590 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272592 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272595 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272597 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272599 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272602 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272605 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272607 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272610 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272612 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:56:24.384603 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272614 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272617 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272622 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272625 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272627 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272630 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272633 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272635 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272638 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272640 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272642 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272645 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272649 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.272652 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273039 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273044 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273048 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273051 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273054 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273057 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:56:24.385544 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273060 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273062 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273065 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273068 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273070 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273075 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273077 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273079 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273082 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273084 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273087 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273106 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273109 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273112 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273117 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273120 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273123 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273125 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273128 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273131 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:56:24.386609 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273134 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273136 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273139 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273141 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273143 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273146 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273148 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273152 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273155 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273158 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273161 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273164 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273167 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273169 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273172 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273175 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273177 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273181 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273184 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273186 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:56:24.387632 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273188 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273191 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273193 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273195 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273198 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273200 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273204 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273206 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273209 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273211 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273213 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273216 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273218 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273221 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273223 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273225 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273228 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273230 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273233 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273237 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:56:24.388617 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273241 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273244 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273246 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273249 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273252 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273254 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273257 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273259 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273262 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273266 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273268 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273271 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273273 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273276 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273278 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273280 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273283 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273285 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273303 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:56:24.389521 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273306 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273310 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273312 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273316 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273320 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273323 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273326 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273328 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273331 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273333 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273336 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273339 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273342 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.273345 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273407 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273413 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273419 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273432 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273437 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273440 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273444 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 16:56:24.390590 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273448 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273453 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273456 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273459 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273462 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273465 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273468 2562 flags.go:64] FLAG: --cgroup-root="" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273471 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273473 2562 flags.go:64] FLAG: --client-ca-file="" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273476 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273480 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273486 2562 flags.go:64] FLAG: --cluster-domain="" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273488 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273492 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273495 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273498 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273502 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273504 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273507 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273511 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273513 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273516 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273519 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273522 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 16:56:24.391612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273525 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273529 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273532 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273535 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273537 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273540 2562 flags.go:64] FLAG: --enable-server="true" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273544 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273548 2562 flags.go:64] FLAG: --event-burst="100" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273551 2562 flags.go:64] FLAG: --event-qps="50" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273555 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273558 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273561 2562 flags.go:64] FLAG: --eviction-hard="" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273564 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273567 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273570 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273573 2562 flags.go:64] FLAG: --eviction-soft="" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273576 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273579 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273582 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273585 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273588 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273591 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273593 2562 flags.go:64] FLAG: --feature-gates="" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273597 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273600 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 16:56:24.392823 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273602 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273605 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273608 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273611 2562 flags.go:64] FLAG: --help="false" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273614 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-134-118.ec2.internal" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273617 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273620 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273622 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273625 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273628 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273631 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273634 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273637 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273640 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273643 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273646 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273650 2562 flags.go:64] FLAG: --kube-reserved="" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273653 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273655 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273659 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273661 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273664 2562 flags.go:64] FLAG: --lock-file="" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273666 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273669 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 16:56:24.394123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273672 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273679 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273682 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273685 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273687 2562 flags.go:64] FLAG: --logging-format="text" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273690 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273694 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273697 2562 flags.go:64] FLAG: --manifest-url="" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273699 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273703 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273706 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273710 2562 flags.go:64] FLAG: --max-pods="110" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273713 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273715 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273719 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273721 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273724 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273727 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273730 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273736 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273739 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273742 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273745 2562 flags.go:64] FLAG: --pod-cidr="" Jul 15 16:56:24.395122 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273748 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273754 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273757 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273760 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273763 2562 flags.go:64] FLAG: --port="10250" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273766 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273768 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e8764b956f6a522f" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273771 2562 flags.go:64] FLAG: --qos-reserved="" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273774 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273777 2562 flags.go:64] FLAG: --register-node="true" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273780 2562 flags.go:64] FLAG: --register-with-taints="" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273785 2562 flags.go:64] FLAG: --registry-burst="10" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273788 2562 flags.go:64] FLAG: --registry-qps="5" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273790 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273793 2562 flags.go:64] FLAG: --reserved-memory="" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273797 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273799 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273802 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273805 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273808 2562 flags.go:64] FLAG: --runonce="false" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273810 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273813 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273816 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273818 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273821 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273824 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 16:56:24.569850 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273827 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273830 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273832 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273835 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273838 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273841 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273844 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273848 2562 flags.go:64] FLAG: --system-cgroups="" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273851 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273856 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273859 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273861 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273866 2562 flags.go:64] FLAG: --tls-min-version="" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273869 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273871 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273874 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273877 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273881 2562 flags.go:64] FLAG: --v="2" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273885 2562 flags.go:64] FLAG: --version="false" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273889 2562 flags.go:64] FLAG: --vmodule="" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273895 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.273899 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274010 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274013 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:56:24.571137 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274016 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274019 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274022 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274025 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274027 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274030 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274033 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274036 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274040 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274043 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274046 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274048 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274051 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274053 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274056 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274060 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274063 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274065 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274068 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:56:24.572793 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274070 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274073 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274076 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274078 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274081 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274083 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274087 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274090 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274092 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274095 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274097 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274100 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274102 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274105 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274107 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274110 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274112 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274114 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274117 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274119 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:56:24.573482 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274121 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274124 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274126 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274129 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274131 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274133 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274136 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274138 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274142 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274145 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274148 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274150 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274152 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274155 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274167 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274170 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274173 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274176 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274180 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274182 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:56:24.574061 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274185 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274187 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274190 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274192 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274195 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274197 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274200 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274202 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274205 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274207 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274210 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274212 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274215 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274217 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274220 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274222 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274225 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274228 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274232 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:56:24.574685 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274235 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274240 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274243 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274246 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274249 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274251 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274254 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274256 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274259 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274261 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274264 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274267 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274270 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274272 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274275 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274277 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274280 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274283 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:56:24.758668 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.274285 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:56:24.759725 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.274949 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 16:56:24.759725 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.281507 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 16:56:24.759725 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.281522 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 16:56:24.759725 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.281540 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 16:56:24.759725 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.281549 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 16:56:24.759725 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281603 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:56:24.759725 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281609 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:56:24.759725 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281612 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:56:24.759725 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281616 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:56:24.759725 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281619 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:56:24.759725 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281622 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:56:24.759725 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281625 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:56:24.759725 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281627 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:56:24.759725 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281630 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281632 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281635 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281637 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281640 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281642 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281645 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281647 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281649 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281652 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281654 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281659 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281662 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281664 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281667 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281671 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281674 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281676 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281678 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281681 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:56:24.760634 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281683 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281687 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281689 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281693 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281697 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281700 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281703 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281706 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281708 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281711 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281714 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281716 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281719 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281722 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281725 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281729 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281732 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281735 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:56:24.761561 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281738 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281740 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281743 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281746 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281749 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281753 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281756 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281758 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281761 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281766 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281769 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281771 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281774 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281776 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281779 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281782 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281786 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281789 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281793 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281796 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:56:24.762705 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281798 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281801 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281803 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281806 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281808 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281811 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281814 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281816 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281818 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281821 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281823 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281826 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281828 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281831 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281833 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281836 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281838 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281842 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281845 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281848 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:56:24.763628 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281850 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:56:24.764562 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281854 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:56:24.764562 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281856 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:56:24.764562 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281859 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:56:24.764562 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281861 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:56:24.764562 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281864 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:56:24.764562 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281867 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:56:24.764562 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281869 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:56:24.764562 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281871 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:56:24.764562 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281874 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:56:24.764562 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281877 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:56:24.764562 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281879 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:56:24.764562 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281882 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:56:24.764562 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.281887 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 16:56:24.764562 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281992 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:56:24.764562 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281996 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.281999 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282002 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282004 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282007 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282009 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282012 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282015 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282017 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282020 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282022 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282026 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282030 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282033 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282038 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282040 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282043 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282046 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282048 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282052 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:56:24.765267 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282055 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282058 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282060 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282063 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282066 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282069 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282071 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282075 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282078 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282081 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282084 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282086 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282089 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282091 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282093 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282096 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282098 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282101 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282103 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:56:24.766206 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282105 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282108 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282110 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282113 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282115 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282118 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282120 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282126 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282128 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282131 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282133 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282136 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282140 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282143 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282145 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282148 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282150 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282153 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282155 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:56:24.767126 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282157 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282160 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282163 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282165 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282168 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282170 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282172 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282175 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282177 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282180 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282182 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282185 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282187 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282189 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282192 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282194 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282196 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282199 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282201 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282204 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:56:24.768048 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282207 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282210 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282212 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282214 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282217 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282221 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282223 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282226 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282228 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282231 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282233 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282236 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282238 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282241 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282243 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282246 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282249 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282251 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282253 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:56:24.768995 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:24.282256 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:56:24.769943 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.282261 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 16:56:24.769943 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.282375 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 16:56:24.769943 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.284395 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 16:56:24.769943 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.285643 2562 server.go:1019] "Starting client certificate rotation" Jul 15 16:56:24.769943 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.285743 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 16:56:24.769943 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.286642 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 16:56:24.769943 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.288188 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 16:56:24.769943 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.291515 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 16:56:24.769943 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.300827 2562 log.go:25] "Validated CRI v1 runtime API" Jul 15 16:56:24.769943 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.303375 2562 log.go:25] "Validated CRI v1 image API" Jul 15 16:56:24.769943 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.304026 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 16:56:24.769943 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.306388 2562 fs.go:135] Filesystem UUIDs: map[3fcf8a42-ea1d-4a9a-8492-e1e6e251e286:/dev/nvme0n1p4 79c26f4c-b031-46b0-8fd9-acf68e70a1ea:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 15 16:56:24.770644 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.306409 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 16:56:24.770873 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.312310 2562 manager.go:216] Machine: {Timestamp:2026-07-15 16:56:24.310916394 +0000 UTC m=+0.359913576 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200680 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ba08fadd401c51750079ca5f2176b SystemUUID:ec2ba08f-add4-01c5-1750-079ca5f2176b BootID:a77684b2-7e21-4b43-9597-c257356e2108 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:96:91:20:b3:f7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:96:91:20:b3:f7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:b8:4a:43:0d:a8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 16:56:24.770873 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.312416 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 16:56:24.770873 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.312483 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 16:56:24.770873 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.316035 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 16:56:24.770873 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.316054 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-118.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 16:56:24.770873 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.316848 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 16:56:24.770873 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.316859 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 16:56:24.770873 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.316866 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.317631 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.319051 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.322134 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.322239 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.322269 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.322286 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.322315 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.322324 2562 kubelet.go:398] "Adding apiserver pod source" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.322332 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.322676 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.322691 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.324137 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.325457 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.327640 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.327666 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.327794 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.327806 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.327813 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.327819 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.327826 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.327831 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.328398 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.328404 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.328410 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.328415 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.328428 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.328436 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.329413 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.329431 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.331453 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.332136 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.332135 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.332868 2562 server.go:1268] "Started kubelet" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.332978 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.332965 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.341608 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.342814 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.342979 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 16:56:24.771784 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.345021 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.348445 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xj5qr" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.349354 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.349371 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.349433 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.349890 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.349908 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.349923 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.350339 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.350349 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.352995 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.356059 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.356168 2562 factory.go:55] Registering systemd factory Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.356198 2562 factory.go:223] Registration of the systemd container factory successfully Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.356434 2562 factory.go:153] Registering CRI-O factory Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.356442 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xj5qr" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.356448 2562 factory.go:223] Registration of the crio container factory successfully Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.356493 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.356517 2562 factory.go:103] Registering Raw factory Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.356528 2562 manager.go:1196] Started watching for new ooms in manager Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.356834 2562 manager.go:349] Starting recovery of all containers Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.362718 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-118.ec2.internal.18c284f3a2a3c62b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-118.ec2.internal,UID:ip-10-0-134-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-118.ec2.internal,},FirstTimestamp:2026-07-15 16:56:24.332699179 +0000 UTC m=+0.381696362,LastTimestamp:2026-07-15 16:56:24.332699179 +0000 UTC m=+0.381696362,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-118.ec2.internal,}" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.366062 2562 manager.go:354] Recovery completed Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.370936 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.373515 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.373543 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.373557 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.374700 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.374711 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.374727 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.374701 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.376024 2562 policy_none.go:49] "None policy: Start" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.376044 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.376055 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 16:56:24.776180 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.376108 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.376139 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.376154 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.377189 2562 policy_none.go:47] "Start" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.418881 2562 manager.go:337] "Starting Device Plugin manager" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.418908 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.418918 2562 server.go:87] "Starting device plugin registration server" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.419146 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.419157 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.419236 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.419345 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.419354 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.419970 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.420020 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-118.ec2.internal\" not found" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.519501 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.520428 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.520461 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.520472 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.520498 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-118.ec2.internal" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.526660 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-118.ec2.internal" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.526679 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-118.ec2.internal\": node \"ip-10-0-134-118.ec2.internal\" not found" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.543447 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.558610 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.560090 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.560138 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.560160 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.560194 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.562351 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.645004 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.661039 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal"] Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.661113 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.662083 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.662113 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.662126 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.663717 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.663831 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.663861 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:56:24.777817 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.664433 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:56:24.784387 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.664464 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:56:24.784387 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.664476 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:56:24.784387 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.664433 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:56:24.784387 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.664546 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:56:24.784387 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.664559 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:56:24.784387 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.666436 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" Jul 15 16:56:24.784387 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.666461 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:56:24.784387 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.667151 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:56:24.784387 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.667169 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:56:24.784387 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.667178 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:56:24.784387 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.695135 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-118.ec2.internal\" not found" node="ip-10-0-134-118.ec2.internal" Jul 15 16:56:24.784387 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.700644 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-118.ec2.internal\" not found" node="ip-10-0-134-118.ec2.internal" Jul 15 16:56:24.784387 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.745703 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jul 15 16:56:24.784387 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.752899 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3c2588fa3cc1960d37ff6f4e9a4674e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal\" (UID: \"f3c2588fa3cc1960d37ff6f4e9a4674e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jul 15 16:56:24.784387 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.752919 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9863c9ecbe102ca15435dceaa71ff34e-config\") pod \"kube-apiserver-proxy-ip-10-0-134-118.ec2.internal\" (UID: \"9863c9ecbe102ca15435dceaa71ff34e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" Jul 15 16:56:24.784387 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.752938 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f3c2588fa3cc1960d37ff6f4e9a4674e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal\" (UID: \"f3c2588fa3cc1960d37ff6f4e9a4674e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jul 15 16:56:24.942566 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.845855 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jul 15 16:56:24.942566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.853833 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f3c2588fa3cc1960d37ff6f4e9a4674e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal\" (UID: \"f3c2588fa3cc1960d37ff6f4e9a4674e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jul 15 16:56:24.942566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.853860 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3c2588fa3cc1960d37ff6f4e9a4674e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal\" (UID: \"f3c2588fa3cc1960d37ff6f4e9a4674e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jul 15 16:56:24.942566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.853879 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9863c9ecbe102ca15435dceaa71ff34e-config\") pod \"kube-apiserver-proxy-ip-10-0-134-118.ec2.internal\" (UID: \"9863c9ecbe102ca15435dceaa71ff34e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" Jul 15 16:56:24.942566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.853919 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9863c9ecbe102ca15435dceaa71ff34e-config\") pod \"kube-apiserver-proxy-ip-10-0-134-118.ec2.internal\" (UID: \"9863c9ecbe102ca15435dceaa71ff34e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" Jul 15 16:56:24.942566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.853939 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f3c2588fa3cc1960d37ff6f4e9a4674e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal\" (UID: \"f3c2588fa3cc1960d37ff6f4e9a4674e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jul 15 16:56:24.942566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:24.853949 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3c2588fa3cc1960d37ff6f4e9a4674e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal\" (UID: \"f3c2588fa3cc1960d37ff6f4e9a4674e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jul 15 16:56:25.115081 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:24.945973 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jul 15 16:56:25.115081 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.000371 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jul 15 16:56:25.115081 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.003459 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" Jul 15 16:56:25.115081 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:25.046950 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jul 15 16:56:25.247168 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:25.148061 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-118.ec2.internal\" not found" Jul 15 16:56:25.247168 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.223912 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:56:25.252238 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.249765 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.258375 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.261280 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.272344 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.285680 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.286961 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.287003 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.324549 2562 apiserver.go:52] "Watching apiserver" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.340432 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.340786 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-8xlk5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh","openshift-cluster-node-tuning-operator/tuned-xn7j8","openshift-dns/node-resolver-tzkgv","openshift-image-registry/node-ca-8w7fh","openshift-multus/network-metrics-daemon-mbqkc","openshift-network-diagnostics/network-check-target-btjr2","openshift-network-operator/iptables-alerter-zmjhz","kube-system/konnectivity-agent-v9jrp","kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal","openshift-multus/multus-6cqnq","openshift-multus/multus-additional-cni-plugins-dx7t2","openshift-ovn-kubernetes/ovnkube-node-wlpcj"] Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.342547 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:25.342639 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.343936 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.345373 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.346268 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qkkz6\"" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.346489 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.346561 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.346989 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tzkgv" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.347372 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.347380 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.348052 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-66hnv\"" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.348318 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.348608 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.348937 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8w7fh" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.349136 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 16:56:25.406523 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.349350 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.349536 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.350425 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:25.356493 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.356580 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8xhf5\"" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.356611 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.356676 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.356695 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.356878 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1fd95390-8b40-475c-ab93-ab641b315e09-host\") pod \"node-ca-8w7fh\" (UID: \"1fd95390-8b40-475c-ab93-ab641b315e09\") " pod="openshift-image-registry/node-ca-8w7fh" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.356908 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1fd95390-8b40-475c-ab93-ab641b315e09-serviceca\") pod \"node-ca-8w7fh\" (UID: \"1fd95390-8b40-475c-ab93-ab641b315e09\") " pod="openshift-image-registry/node-ca-8w7fh" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.356930 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zdhrx\"" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.356932 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76a7ad68-9351-4709-b119-ac5a58ca509e-cni-binary-copy\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.356993 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.357008 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cc6a2e63-80e9-44d9-9708-dc85bd571957-hosts-file\") pod \"node-resolver-tzkgv\" (UID: \"cc6a2e63-80e9-44d9-9708-dc85bd571957\") " pod="openshift-dns/node-resolver-tzkgv" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.357037 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc6a2e63-80e9-44d9-9708-dc85bd571957-tmp-dir\") pod \"node-resolver-tzkgv\" (UID: \"cc6a2e63-80e9-44d9-9708-dc85bd571957\") " pod="openshift-dns/node-resolver-tzkgv" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.357097 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dgnv\" (UniqueName: \"kubernetes.io/projected/cc6a2e63-80e9-44d9-9708-dc85bd571957-kube-api-access-9dgnv\") pod \"node-resolver-tzkgv\" (UID: \"cc6a2e63-80e9-44d9-9708-dc85bd571957\") " pod="openshift-dns/node-resolver-tzkgv" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.357125 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6thjk\" (UniqueName: \"kubernetes.io/projected/1fd95390-8b40-475c-ab93-ab641b315e09-kube-api-access-6thjk\") pod \"node-ca-8w7fh\" (UID: \"1fd95390-8b40-475c-ab93-ab641b315e09\") " pod="openshift-image-registry/node-ca-8w7fh" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.357149 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs\") pod \"network-metrics-daemon-mbqkc\" (UID: \"4f43f08e-aa70-42d2-9207-93156d956d7f\") " pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:25.408083 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.357172 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8chpd\" (UniqueName: \"kubernetes.io/projected/4f43f08e-aa70-42d2-9207-93156d956d7f-kube-api-access-8chpd\") pod \"network-metrics-daemon-mbqkc\" (UID: \"4f43f08e-aa70-42d2-9207-93156d956d7f\") " pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.357199 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/76a7ad68-9351-4709-b119-ac5a58ca509e-os-release\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.357224 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76a7ad68-9351-4709-b119-ac5a58ca509e-cnibin\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.357249 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2kmc\" (UniqueName: \"kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc\") pod \"network-check-target-btjr2\" (UID: \"8ba9f582-94a1-4bdf-b4de-6047b45368d2\") " pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.357274 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76a7ad68-9351-4709-b119-ac5a58ca509e-system-cni-dir\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.357994 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v9jrp" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.359323 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:25.359376 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.360109 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.360400 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.360426 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2stph\"" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.360777 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 16:51:24 +0000 UTC" deadline="2028-02-03 03:25:47.420106147 +0000 UTC" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.360801 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.360806 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13618h29m22.059302521s" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.362628 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.363450 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.363512 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.363679 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.363756 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9p622\"" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.364122 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zmjhz" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.365103 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.365103 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.365178 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.365187 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 16:56:25.410732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.365230 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pdknh\"" Jul 15 16:56:25.411861 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.365705 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.411861 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.366234 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 16:56:25.411861 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.366302 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 16:56:25.411861 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.367574 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 16:56:25.411861 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.367672 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 16:56:25.411861 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.367788 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-727ht\"" Jul 15 16:56:25.411861 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.368236 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 16:56:25.411861 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.368334 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-g9xvl\"" Jul 15 16:56:25.411861 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.368537 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 16:56:25.411861 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.377466 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 16:56:25.411861 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.411338 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wbh54" Jul 15 16:56:25.536703 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.423192 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wbh54" Jul 15 16:56:25.536703 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.456054 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 16:56:25.536703 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457413 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76a7ad68-9351-4709-b119-ac5a58ca509e-cnibin\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.536703 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457453 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-sysctl-d\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.536703 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457478 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76a7ad68-9351-4709-b119-ac5a58ca509e-cnibin\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.536703 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457488 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-run-systemd\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.536703 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457534 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-var-lib-openvswitch\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.536703 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457553 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-run-ovn\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.536703 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457572 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-run-ovn-kubernetes\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.536703 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457587 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-cnibin\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.536703 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457642 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5rfn\" (UniqueName: \"kubernetes.io/projected/59101d7a-bd9b-4db5-b05b-4466aa58c6c5-kube-api-access-p5rfn\") pod \"iptables-alerter-zmjhz\" (UID: \"59101d7a-bd9b-4db5-b05b-4466aa58c6c5\") " pod="openshift-network-operator/iptables-alerter-zmjhz" Jul 15 16:56:25.536703 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457666 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/76a7ad68-9351-4709-b119-ac5a58ca509e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.536703 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457682 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-system-cni-dir\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.536703 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457700 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dgnv\" (UniqueName: \"kubernetes.io/projected/cc6a2e63-80e9-44d9-9708-dc85bd571957-kube-api-access-9dgnv\") pod \"node-resolver-tzkgv\" (UID: \"cc6a2e63-80e9-44d9-9708-dc85bd571957\") " pod="openshift-dns/node-resolver-tzkgv" Jul 15 16:56:25.536703 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457716 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs\") pod \"network-metrics-daemon-mbqkc\" (UID: \"4f43f08e-aa70-42d2-9207-93156d956d7f\") " pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:25.537722 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457731 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-run-netns\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.537722 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457745 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-sysctl-conf\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.537722 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457759 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-lib-modules\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.537722 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457779 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.537722 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457802 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-os-release\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.537722 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457837 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2kmc\" (UniqueName: \"kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc\") pod \"network-check-target-btjr2\" (UID: \"8ba9f582-94a1-4bdf-b4de-6047b45368d2\") " pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:25.537722 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457888 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76a7ad68-9351-4709-b119-ac5a58ca509e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.537722 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:25.457941 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:25.537722 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.457984 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-modprobe-d\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.537722 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458009 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-var-lib-kubelet\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.537722 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458066 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-run-k8s-cni-cncf-io\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.537722 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458113 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-run-netns\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.537722 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:25.458170 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs podName:4f43f08e-aa70-42d2-9207-93156d956d7f nodeName:}" failed. No retries permitted until 2026-07-15 16:56:25.958130669 +0000 UTC m=+2.007127846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs") pod "network-metrics-daemon-mbqkc" (UID: "4f43f08e-aa70-42d2-9207-93156d956d7f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:25.538681 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458191 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/375d7469-6ffd-4be1-94bf-2782015de14d-multus-daemon-config\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.538681 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458210 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-etc-kubernetes\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.538681 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458232 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1fd95390-8b40-475c-ab93-ab641b315e09-host\") pod \"node-ca-8w7fh\" (UID: \"1fd95390-8b40-475c-ab93-ab641b315e09\") " pod="openshift-image-registry/node-ca-8w7fh" Jul 15 16:56:25.538681 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458249 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/76a7ad68-9351-4709-b119-ac5a58ca509e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.538681 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458271 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-host\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.538681 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458295 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-cni-bin\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.538681 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458329 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1fd95390-8b40-475c-ab93-ab641b315e09-host\") pod \"node-ca-8w7fh\" (UID: \"1fd95390-8b40-475c-ab93-ab641b315e09\") " pod="openshift-image-registry/node-ca-8w7fh" Jul 15 16:56:25.538681 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458339 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-ovnkube-script-lib\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.538681 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458369 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-etc-selinux\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.538681 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458385 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cc6a2e63-80e9-44d9-9708-dc85bd571957-hosts-file\") pod \"node-resolver-tzkgv\" (UID: \"cc6a2e63-80e9-44d9-9708-dc85bd571957\") " pod="openshift-dns/node-resolver-tzkgv" Jul 15 16:56:25.538681 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458400 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d626712c-c088-4ffe-b923-0b440dcc79d1-tmp\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.538681 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458415 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blg2z\" (UniqueName: \"kubernetes.io/projected/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-kube-api-access-blg2z\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.538681 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458462 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cc6a2e63-80e9-44d9-9708-dc85bd571957-hosts-file\") pod \"node-resolver-tzkgv\" (UID: \"cc6a2e63-80e9-44d9-9708-dc85bd571957\") " pod="openshift-dns/node-resolver-tzkgv" Jul 15 16:56:25.538681 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458490 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.539413 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458509 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-sys-fs\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.539413 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458524 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-sysconfig\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.539413 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458538 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-kubernetes\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.539413 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458552 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-etc-openvswitch\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.539413 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458586 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-log-socket\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.539413 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458611 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-cni-netd\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.539413 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458645 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-var-lib-kubelet\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.539413 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458661 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-slash\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.539413 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458675 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/375d7469-6ffd-4be1-94bf-2782015de14d-cni-binary-copy\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.539413 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458689 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/abe0c218-237b-4de9-88bb-9c2623db9bfa-konnectivity-ca\") pod \"konnectivity-agent-v9jrp\" (UID: \"abe0c218-237b-4de9-88bb-9c2623db9bfa\") " pod="kube-system/konnectivity-agent-v9jrp" Jul 15 16:56:25.539413 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458705 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1356768b-d3b0-4d99-98c7-42a27e4d0818-kubelet-config\") pod \"global-pull-secret-syncer-8xlk5\" (UID: \"1356768b-d3b0-4d99-98c7-42a27e4d0818\") " pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:25.539413 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458726 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76a7ad68-9351-4709-b119-ac5a58ca509e-cni-binary-copy\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.539413 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458758 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-systemd\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.539413 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458773 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-sys\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.540137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458788 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-run-openvswitch\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.540137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458809 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-device-dir\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.540137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458825 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc6a2e63-80e9-44d9-9708-dc85bd571957-tmp-dir\") pod \"node-resolver-tzkgv\" (UID: \"cc6a2e63-80e9-44d9-9708-dc85bd571957\") " pod="openshift-dns/node-resolver-tzkgv" Jul 15 16:56:25.540137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458840 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8chpd\" (UniqueName: \"kubernetes.io/projected/4f43f08e-aa70-42d2-9207-93156d956d7f-kube-api-access-8chpd\") pod \"network-metrics-daemon-mbqkc\" (UID: \"4f43f08e-aa70-42d2-9207-93156d956d7f\") " pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:25.540137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458857 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnt77\" (UniqueName: \"kubernetes.io/projected/76a7ad68-9351-4709-b119-ac5a58ca509e-kube-api-access-hnt77\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.540137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458882 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-kubelet\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.540137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458906 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-var-lib-cni-bin\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.540137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458930 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-var-lib-cni-multus\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.540137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458952 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-socket-dir\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.540137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.458982 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9pkd\" (UniqueName: \"kubernetes.io/projected/aac4144b-3d96-41bc-b02f-d2f5c39acc81-kube-api-access-n9pkd\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.540137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459029 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/76a7ad68-9351-4709-b119-ac5a58ca509e-os-release\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.540137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459052 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-run\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.540137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459075 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-tuned\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.540137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459096 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/76a7ad68-9351-4709-b119-ac5a58ca509e-os-release\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.542940 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459103 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-node-log\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.542940 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459120 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-env-overrides\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.542940 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459135 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-run-multus-certs\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.542940 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459151 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/59101d7a-bd9b-4db5-b05b-4466aa58c6c5-iptables-alerter-script\") pod \"iptables-alerter-zmjhz\" (UID: \"59101d7a-bd9b-4db5-b05b-4466aa58c6c5\") " pod="openshift-network-operator/iptables-alerter-zmjhz" Jul 15 16:56:25.542940 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459168 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76a7ad68-9351-4709-b119-ac5a58ca509e-system-cni-dir\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.542940 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459199 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-hostroot\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.542940 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459212 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76a7ad68-9351-4709-b119-ac5a58ca509e-system-cni-dir\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.542940 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459220 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-multus-conf-dir\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.542940 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459235 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nj76\" (UniqueName: \"kubernetes.io/projected/375d7469-6ffd-4be1-94bf-2782015de14d-kube-api-access-2nj76\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.542940 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459251 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/abe0c218-237b-4de9-88bb-9c2623db9bfa-agent-certs\") pod \"konnectivity-agent-v9jrp\" (UID: \"abe0c218-237b-4de9-88bb-9c2623db9bfa\") " pod="kube-system/konnectivity-agent-v9jrp" Jul 15 16:56:25.542940 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459270 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/59101d7a-bd9b-4db5-b05b-4466aa58c6c5-host-slash\") pod \"iptables-alerter-zmjhz\" (UID: \"59101d7a-bd9b-4db5-b05b-4466aa58c6c5\") " pod="openshift-network-operator/iptables-alerter-zmjhz" Jul 15 16:56:25.542940 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459301 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1fd95390-8b40-475c-ab93-ab641b315e09-serviceca\") pod \"node-ca-8w7fh\" (UID: \"1fd95390-8b40-475c-ab93-ab641b315e09\") " pod="openshift-image-registry/node-ca-8w7fh" Jul 15 16:56:25.542940 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459331 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-ovnkube-config\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.542940 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459425 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6thjk\" (UniqueName: \"kubernetes.io/projected/1fd95390-8b40-475c-ab93-ab641b315e09-kube-api-access-6thjk\") pod \"node-ca-8w7fh\" (UID: \"1fd95390-8b40-475c-ab93-ab641b315e09\") " pod="openshift-image-registry/node-ca-8w7fh" Jul 15 16:56:25.543891 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459450 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-ovn-node-metrics-cert\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.543891 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459545 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-multus-cni-dir\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.543891 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459560 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-registration-dir\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.543891 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459576 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfm5g\" (UniqueName: \"kubernetes.io/projected/d626712c-c088-4ffe-b923-0b440dcc79d1-kube-api-access-sfm5g\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.543891 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459594 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-systemd-units\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.543891 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459617 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-multus-socket-dir-parent\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.543891 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459632 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1356768b-d3b0-4d99-98c7-42a27e4d0818-dbus\") pod \"global-pull-secret-syncer-8xlk5\" (UID: \"1356768b-d3b0-4d99-98c7-42a27e4d0818\") " pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:25.543891 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.459650 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret\") pod \"global-pull-secret-syncer-8xlk5\" (UID: \"1356768b-d3b0-4d99-98c7-42a27e4d0818\") " pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:25.543891 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.461588 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc6a2e63-80e9-44d9-9708-dc85bd571957-tmp-dir\") pod \"node-resolver-tzkgv\" (UID: \"cc6a2e63-80e9-44d9-9708-dc85bd571957\") " pod="openshift-dns/node-resolver-tzkgv" Jul 15 16:56:25.543891 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.462091 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1fd95390-8b40-475c-ab93-ab641b315e09-serviceca\") pod \"node-ca-8w7fh\" (UID: \"1fd95390-8b40-475c-ab93-ab641b315e09\") " pod="openshift-image-registry/node-ca-8w7fh" Jul 15 16:56:25.543891 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:25.470558 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:56:25.543891 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:25.470573 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:56:25.543891 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:25.470582 2562 projected.go:196] Error preparing data for projected volume kube-api-access-q2kmc for pod openshift-network-diagnostics/network-check-target-btjr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:25.543891 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:25.470629 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc podName:8ba9f582-94a1-4bdf-b4de-6047b45368d2 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:25.97061594 +0000 UTC m=+2.019613128 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-q2kmc" (UniqueName: "kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc") pod "network-check-target-btjr2" (UID: "8ba9f582-94a1-4bdf-b4de-6047b45368d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:25.544612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.470890 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 16:56:25.544612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.473010 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76a7ad68-9351-4709-b119-ac5a58ca509e-cni-binary-copy\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.544612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.474077 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8chpd\" (UniqueName: \"kubernetes.io/projected/4f43f08e-aa70-42d2-9207-93156d956d7f-kube-api-access-8chpd\") pod \"network-metrics-daemon-mbqkc\" (UID: \"4f43f08e-aa70-42d2-9207-93156d956d7f\") " pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:25.544612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.474082 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6thjk\" (UniqueName: \"kubernetes.io/projected/1fd95390-8b40-475c-ab93-ab641b315e09-kube-api-access-6thjk\") pod \"node-ca-8w7fh\" (UID: \"1fd95390-8b40-475c-ab93-ab641b315e09\") " pod="openshift-image-registry/node-ca-8w7fh" Jul 15 16:56:25.544612 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.474079 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dgnv\" (UniqueName: \"kubernetes.io/projected/cc6a2e63-80e9-44d9-9708-dc85bd571957-kube-api-access-9dgnv\") pod \"node-resolver-tzkgv\" (UID: \"cc6a2e63-80e9-44d9-9708-dc85bd571957\") " pod="openshift-dns/node-resolver-tzkgv" Jul 15 16:56:25.648566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.559848 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfm5g\" (UniqueName: \"kubernetes.io/projected/d626712c-c088-4ffe-b923-0b440dcc79d1-kube-api-access-sfm5g\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.648566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.559873 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-systemd-units\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.648566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.559890 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-multus-socket-dir-parent\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.648566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.559905 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1356768b-d3b0-4d99-98c7-42a27e4d0818-dbus\") pod \"global-pull-secret-syncer-8xlk5\" (UID: \"1356768b-d3b0-4d99-98c7-42a27e4d0818\") " pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:25.648566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.559922 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret\") pod \"global-pull-secret-syncer-8xlk5\" (UID: \"1356768b-d3b0-4d99-98c7-42a27e4d0818\") " pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:25.648566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.559944 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-sysctl-d\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.648566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.559966 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-run-systemd\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.648566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.559962 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-systemd-units\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.648566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.559990 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-var-lib-openvswitch\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.648566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560015 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-run-ovn\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.648566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560035 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-var-lib-openvswitch\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.648566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560037 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-run-ovn-kubernetes\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.648566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560046 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-multus-socket-dir-parent\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.648566 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:25.560051 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:25.648566 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560069 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-run-systemd\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.649995 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560090 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1356768b-d3b0-4d99-98c7-42a27e4d0818-dbus\") pod \"global-pull-secret-syncer-8xlk5\" (UID: \"1356768b-d3b0-4d99-98c7-42a27e4d0818\") " pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:25.649995 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560096 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-run-ovn-kubernetes\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.649995 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560106 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-run-ovn\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.649995 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560120 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-cnibin\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.649995 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:25.560164 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret podName:1356768b-d3b0-4d99-98c7-42a27e4d0818 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:26.060150338 +0000 UTC m=+2.109147508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret") pod "global-pull-secret-syncer-8xlk5" (UID: "1356768b-d3b0-4d99-98c7-42a27e4d0818") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:25.649995 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560176 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-sysctl-d\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.649995 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5rfn\" (UniqueName: \"kubernetes.io/projected/59101d7a-bd9b-4db5-b05b-4466aa58c6c5-kube-api-access-p5rfn\") pod \"iptables-alerter-zmjhz\" (UID: \"59101d7a-bd9b-4db5-b05b-4466aa58c6c5\") " pod="openshift-network-operator/iptables-alerter-zmjhz" Jul 15 16:56:25.649995 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560245 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-cnibin\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.649995 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560243 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/76a7ad68-9351-4709-b119-ac5a58ca509e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.649995 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560282 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-system-cni-dir\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.649995 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560335 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-run-netns\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.649995 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560350 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-sysctl-conf\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.649995 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560365 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-system-cni-dir\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.649995 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560369 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-lib-modules\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.650925 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560396 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-run-netns\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.650925 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560414 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.650925 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560444 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-os-release\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.650925 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560455 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.650925 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560463 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-sysctl-conf\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.650925 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560467 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-lib-modules\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.650925 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560482 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76a7ad68-9351-4709-b119-ac5a58ca509e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.650925 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560505 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-modprobe-d\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.650925 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560511 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-os-release\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.650925 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560520 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-var-lib-kubelet\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.650925 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560539 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-run-k8s-cni-cncf-io\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.650925 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560565 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-run-netns\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.650925 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560576 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-var-lib-kubelet\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.650925 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560588 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-run-k8s-cni-cncf-io\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.652077 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560592 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/375d7469-6ffd-4be1-94bf-2782015de14d-multus-daemon-config\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.652077 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560569 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-modprobe-d\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.652077 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560616 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-run-netns\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.652077 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560619 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-etc-kubernetes\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.652077 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560595 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76a7ad68-9351-4709-b119-ac5a58ca509e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.652077 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560651 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-etc-kubernetes\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.652077 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560650 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/76a7ad68-9351-4709-b119-ac5a58ca509e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.652077 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-host\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.652077 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560712 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-cni-bin\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.652077 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560721 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/76a7ad68-9351-4709-b119-ac5a58ca509e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.652077 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560735 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-ovnkube-script-lib\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.652077 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560775 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-host\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.652077 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560801 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-cni-bin\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.652077 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560821 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-etc-selinux\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.652077 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560852 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d626712c-c088-4ffe-b923-0b440dcc79d1-tmp\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.653033 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560876 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blg2z\" (UniqueName: \"kubernetes.io/projected/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-kube-api-access-blg2z\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.653033 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560899 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.653033 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560923 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-sys-fs\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.653033 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560947 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-sysconfig\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.653033 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560948 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-etc-selinux\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.653033 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560959 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.653033 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.560998 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-sysconfig\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.653033 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561003 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-sys-fs\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.653033 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561007 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-kubernetes\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.653033 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561062 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-etc-openvswitch\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.653033 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561089 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-log-socket\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.653033 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561114 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-cni-netd\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.653033 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561115 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-kubernetes\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.653033 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561138 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-var-lib-kubelet\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.653980 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561143 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/76a7ad68-9351-4709-b119-ac5a58ca509e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.653980 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561161 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-slash\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.653980 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561169 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-cni-netd\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.653980 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561173 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-log-socket\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.653980 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561196 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/375d7469-6ffd-4be1-94bf-2782015de14d-cni-binary-copy\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.653980 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561201 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-slash\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.653980 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561207 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-etc-openvswitch\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.653980 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561202 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-var-lib-kubelet\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.653980 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561249 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/abe0c218-237b-4de9-88bb-9c2623db9bfa-konnectivity-ca\") pod \"konnectivity-agent-v9jrp\" (UID: \"abe0c218-237b-4de9-88bb-9c2623db9bfa\") " pod="kube-system/konnectivity-agent-v9jrp" Jul 15 16:56:25.653980 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1356768b-d3b0-4d99-98c7-42a27e4d0818-kubelet-config\") pod \"global-pull-secret-syncer-8xlk5\" (UID: \"1356768b-d3b0-4d99-98c7-42a27e4d0818\") " pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:25.653980 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561290 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/375d7469-6ffd-4be1-94bf-2782015de14d-multus-daemon-config\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.653980 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561305 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-systemd\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.653980 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561306 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-ovnkube-script-lib\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.653980 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561345 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-sys\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.653980 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561374 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1356768b-d3b0-4d99-98c7-42a27e4d0818-kubelet-config\") pod \"global-pull-secret-syncer-8xlk5\" (UID: \"1356768b-d3b0-4d99-98c7-42a27e4d0818\") " pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:25.654901 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561388 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-systemd\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.654901 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561391 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-run-openvswitch\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.654901 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-device-dir\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.654901 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561431 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-sys\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.654901 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561424 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-run-openvswitch\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.654901 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561459 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnt77\" (UniqueName: \"kubernetes.io/projected/76a7ad68-9351-4709-b119-ac5a58ca509e-kube-api-access-hnt77\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.654901 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561487 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-kubelet\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.654901 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561483 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-device-dir\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.654901 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561544 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-var-lib-cni-bin\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.654901 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561558 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-host-kubelet\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.654901 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561570 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-var-lib-cni-multus\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.654901 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561596 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-socket-dir\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.654901 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561624 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9pkd\" (UniqueName: \"kubernetes.io/projected/aac4144b-3d96-41bc-b02f-d2f5c39acc81-kube-api-access-n9pkd\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.654901 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561634 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-var-lib-cni-bin\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.655616 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561637 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-var-lib-cni-multus\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.655616 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561650 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-run\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.655616 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561682 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-tuned\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.655616 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561685 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/375d7469-6ffd-4be1-94bf-2782015de14d-cni-binary-copy\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.655616 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561714 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-node-log\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.655616 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561743 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-socket-dir\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.655616 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561762 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-env-overrides\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.655616 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561733 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d626712c-c088-4ffe-b923-0b440dcc79d1-run\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.655616 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561791 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-run-multus-certs\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.655616 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561797 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-node-log\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.655616 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561811 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/abe0c218-237b-4de9-88bb-9c2623db9bfa-konnectivity-ca\") pod \"konnectivity-agent-v9jrp\" (UID: \"abe0c218-237b-4de9-88bb-9c2623db9bfa\") " pod="kube-system/konnectivity-agent-v9jrp" Jul 15 16:56:25.655616 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561827 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-host-run-multus-certs\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.655616 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/59101d7a-bd9b-4db5-b05b-4466aa58c6c5-iptables-alerter-script\") pod \"iptables-alerter-zmjhz\" (UID: \"59101d7a-bd9b-4db5-b05b-4466aa58c6c5\") " pod="openshift-network-operator/iptables-alerter-zmjhz" Jul 15 16:56:25.655616 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561879 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-hostroot\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.655616 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561908 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-multus-conf-dir\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.656772 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561946 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-multus-conf-dir\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.656772 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561954 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-hostroot\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.656772 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.561997 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nj76\" (UniqueName: \"kubernetes.io/projected/375d7469-6ffd-4be1-94bf-2782015de14d-kube-api-access-2nj76\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.656772 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.562024 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/abe0c218-237b-4de9-88bb-9c2623db9bfa-agent-certs\") pod \"konnectivity-agent-v9jrp\" (UID: \"abe0c218-237b-4de9-88bb-9c2623db9bfa\") " pod="kube-system/konnectivity-agent-v9jrp" Jul 15 16:56:25.656772 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.562045 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/59101d7a-bd9b-4db5-b05b-4466aa58c6c5-host-slash\") pod \"iptables-alerter-zmjhz\" (UID: \"59101d7a-bd9b-4db5-b05b-4466aa58c6c5\") " pod="openshift-network-operator/iptables-alerter-zmjhz" Jul 15 16:56:25.656772 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.562117 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/59101d7a-bd9b-4db5-b05b-4466aa58c6c5-host-slash\") pod \"iptables-alerter-zmjhz\" (UID: \"59101d7a-bd9b-4db5-b05b-4466aa58c6c5\") " pod="openshift-network-operator/iptables-alerter-zmjhz" Jul 15 16:56:25.656772 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.562155 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-ovnkube-config\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.656772 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.562205 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-ovn-node-metrics-cert\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.656772 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.562223 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-env-overrides\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.656772 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.562234 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-multus-cni-dir\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.656772 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.562324 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/375d7469-6ffd-4be1-94bf-2782015de14d-multus-cni-dir\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.656772 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.562328 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-registration-dir\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.656772 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.562430 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aac4144b-3d96-41bc-b02f-d2f5c39acc81-registration-dir\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.656772 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.562588 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/59101d7a-bd9b-4db5-b05b-4466aa58c6c5-iptables-alerter-script\") pod \"iptables-alerter-zmjhz\" (UID: \"59101d7a-bd9b-4db5-b05b-4466aa58c6c5\") " pod="openshift-network-operator/iptables-alerter-zmjhz" Jul 15 16:56:25.656772 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.562650 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-ovnkube-config\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.657481 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.563809 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d626712c-c088-4ffe-b923-0b440dcc79d1-tmp\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.657481 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.569327 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d626712c-c088-4ffe-b923-0b440dcc79d1-etc-tuned\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.657481 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.569531 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/abe0c218-237b-4de9-88bb-9c2623db9bfa-agent-certs\") pod \"konnectivity-agent-v9jrp\" (UID: \"abe0c218-237b-4de9-88bb-9c2623db9bfa\") " pod="kube-system/konnectivity-agent-v9jrp" Jul 15 16:56:25.657481 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.569638 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-ovn-node-metrics-cert\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.657481 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.572591 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blg2z\" (UniqueName: \"kubernetes.io/projected/e8547271-3ba8-4a2c-8e1e-d3c18de88bc2-kube-api-access-blg2z\") pod \"ovnkube-node-wlpcj\" (UID: \"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.657481 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.574658 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9pkd\" (UniqueName: \"kubernetes.io/projected/aac4144b-3d96-41bc-b02f-d2f5c39acc81-kube-api-access-n9pkd\") pod \"aws-ebs-csi-driver-node-x7bdh\" (UID: \"aac4144b-3d96-41bc-b02f-d2f5c39acc81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.657481 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.574784 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfm5g\" (UniqueName: \"kubernetes.io/projected/d626712c-c088-4ffe-b923-0b440dcc79d1-kube-api-access-sfm5g\") pod \"tuned-xn7j8\" (UID: \"d626712c-c088-4ffe-b923-0b440dcc79d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.657481 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.576231 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nj76\" (UniqueName: \"kubernetes.io/projected/375d7469-6ffd-4be1-94bf-2782015de14d-kube-api-access-2nj76\") pod \"multus-6cqnq\" (UID: \"375d7469-6ffd-4be1-94bf-2782015de14d\") " pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.657481 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.576729 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5rfn\" (UniqueName: \"kubernetes.io/projected/59101d7a-bd9b-4db5-b05b-4466aa58c6c5-kube-api-access-p5rfn\") pod \"iptables-alerter-zmjhz\" (UID: \"59101d7a-bd9b-4db5-b05b-4466aa58c6c5\") " pod="openshift-network-operator/iptables-alerter-zmjhz" Jul 15 16:56:25.657481 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.577738 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnt77\" (UniqueName: \"kubernetes.io/projected/76a7ad68-9351-4709-b119-ac5a58ca509e-kube-api-access-hnt77\") pod \"multus-additional-cni-plugins-dx7t2\" (UID: \"76a7ad68-9351-4709-b119-ac5a58ca509e\") " pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.657481 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.588858 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zmjhz" Jul 15 16:56:25.657481 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.608774 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6cqnq" Jul 15 16:56:25.789739 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.677218 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dx7t2" Jul 15 16:56:25.789739 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.682698 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:56:25.789739 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.696793 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" Jul 15 16:56:25.789739 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.716309 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:56:25.789739 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.733724 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tzkgv" Jul 15 16:56:25.789739 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.763401 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8w7fh" Jul 15 16:56:25.789739 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.787397 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v9jrp" Jul 15 16:56:25.905212 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.809207 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" Jul 15 16:56:25.905212 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.823799 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:25.905212 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:25.859608 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaac4144b_3d96_41bc_b02f_d2f5c39acc81.slice/crio-fec84568c73a1c0ea16cd9decd8ec60e48da059347e1e224e3e67accc7f2b820 WatchSource:0}: Error finding container fec84568c73a1c0ea16cd9decd8ec60e48da059347e1e224e3e67accc7f2b820: Status 404 returned error can't find the container with id fec84568c73a1c0ea16cd9decd8ec60e48da059347e1e224e3e67accc7f2b820 Jul 15 16:56:25.905212 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:25.865873 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9863c9ecbe102ca15435dceaa71ff34e.slice/crio-03afd28c67a60a24c7d167597251c521de4c60711bec585b5f8af0c11615d399 WatchSource:0}: Error finding container 03afd28c67a60a24c7d167597251c521de4c60711bec585b5f8af0c11615d399: Status 404 returned error can't find the container with id 03afd28c67a60a24c7d167597251c521de4c60711bec585b5f8af0c11615d399 Jul 15 16:56:25.905212 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:25.866860 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8547271_3ba8_4a2c_8e1e_d3c18de88bc2.slice/crio-b0bfaa43a5adbe38c56371630611fd9a83cd1ba3b911886db444677b49cb1378 WatchSource:0}: Error finding container b0bfaa43a5adbe38c56371630611fd9a83cd1ba3b911886db444677b49cb1378: Status 404 returned error can't find the container with id b0bfaa43a5adbe38c56371630611fd9a83cd1ba3b911886db444677b49cb1378 Jul 15 16:56:25.905212 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.873207 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 16:56:25.905212 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:25.874086 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3c2588fa3cc1960d37ff6f4e9a4674e.slice/crio-02986b1a9d19c3f3d4d4e54be461b8f06d9985c781d46b1409cb6fa9dc13f0dc WatchSource:0}: Error finding container 02986b1a9d19c3f3d4d4e54be461b8f06d9985c781d46b1409cb6fa9dc13f0dc: Status 404 returned error can't find the container with id 02986b1a9d19c3f3d4d4e54be461b8f06d9985c781d46b1409cb6fa9dc13f0dc Jul 15 16:56:26.025014 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:25.964543 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs\") pod \"network-metrics-daemon-mbqkc\" (UID: \"4f43f08e-aa70-42d2-9207-93156d956d7f\") " pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:26.025014 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:25.964676 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:26.025014 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:25.964744 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs podName:4f43f08e-aa70-42d2-9207-93156d956d7f nodeName:}" failed. No retries permitted until 2026-07-15 16:56:26.964727935 +0000 UTC m=+3.013725111 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs") pod "network-metrics-daemon-mbqkc" (UID: "4f43f08e-aa70-42d2-9207-93156d956d7f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:26.025014 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:25.995731 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59101d7a_bd9b_4db5_b05b_4466aa58c6c5.slice/crio-6968c14cce1db6084676b350d9ee78dd57be39985c7637da11423b0f2cfcd97d WatchSource:0}: Error finding container 6968c14cce1db6084676b350d9ee78dd57be39985c7637da11423b0f2cfcd97d: Status 404 returned error can't find the container with id 6968c14cce1db6084676b350d9ee78dd57be39985c7637da11423b0f2cfcd97d Jul 15 16:56:26.025014 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:26.024780 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76a7ad68_9351_4709_b119_ac5a58ca509e.slice/crio-854ca8378b2284cc025d9cb735a35070ce03f161af7cb93840c935652090c2d2 WatchSource:0}: Error finding container 854ca8378b2284cc025d9cb735a35070ce03f161af7cb93840c935652090c2d2: Status 404 returned error can't find the container with id 854ca8378b2284cc025d9cb735a35070ce03f161af7cb93840c935652090c2d2 Jul 15 16:56:26.198580 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:26.042080 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod375d7469_6ffd_4be1_94bf_2782015de14d.slice/crio-fb99c92895aeaa6ca3e7eb0a8ba4b989d4e5d7271d753107fde99ec293e35789 WatchSource:0}: Error finding container fb99c92895aeaa6ca3e7eb0a8ba4b989d4e5d7271d753107fde99ec293e35789: Status 404 returned error can't find the container with id fb99c92895aeaa6ca3e7eb0a8ba4b989d4e5d7271d753107fde99ec293e35789 Jul 15 16:56:26.198580 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.065435 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret\") pod \"global-pull-secret-syncer-8xlk5\" (UID: \"1356768b-d3b0-4d99-98c7-42a27e4d0818\") " pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:26.198580 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:26.065563 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:26.198580 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:26.065626 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret podName:1356768b-d3b0-4d99-98c7-42a27e4d0818 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:27.065607704 +0000 UTC m=+3.114604878 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret") pod "global-pull-secret-syncer-8xlk5" (UID: "1356768b-d3b0-4d99-98c7-42a27e4d0818") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:26.198580 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.065927 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2kmc\" (UniqueName: \"kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc\") pod \"network-check-target-btjr2\" (UID: \"8ba9f582-94a1-4bdf-b4de-6047b45368d2\") " pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:26.198580 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:26.066009 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:56:26.198580 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:26.066027 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:56:26.198580 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:26.066037 2562 projected.go:196] Error preparing data for projected volume kube-api-access-q2kmc for pod openshift-network-diagnostics/network-check-target-btjr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:26.198580 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:26.066083 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc podName:8ba9f582-94a1-4bdf-b4de-6047b45368d2 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:27.06606671 +0000 UTC m=+3.115063881 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-q2kmc" (UniqueName: "kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc") pod "network-check-target-btjr2" (UID: "8ba9f582-94a1-4bdf-b4de-6047b45368d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:26.198580 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:26.148088 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc6a2e63_80e9_44d9_9708_dc85bd571957.slice/crio-2e278d79d2a3ca75a61769a92e0276334f27d73d000ee889343c60fb2de4c2c7 WatchSource:0}: Error finding container 2e278d79d2a3ca75a61769a92e0276334f27d73d000ee889343c60fb2de4c2c7: Status 404 returned error can't find the container with id 2e278d79d2a3ca75a61769a92e0276334f27d73d000ee889343c60fb2de4c2c7 Jul 15 16:56:26.198580 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.182194 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:56:26.316592 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:26.249998 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd626712c_c088_4ffe_b923_0b440dcc79d1.slice/crio-60b4acb02f26f859ef43476d0992bedb5bfe10b4faea5999a377ddcf9f4ad2b8 WatchSource:0}: Error finding container 60b4acb02f26f859ef43476d0992bedb5bfe10b4faea5999a377ddcf9f4ad2b8: Status 404 returned error can't find the container with id 60b4acb02f26f859ef43476d0992bedb5bfe10b4faea5999a377ddcf9f4ad2b8 Jul 15 16:56:26.425995 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.425875 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 16:51:25 +0000 UTC" deadline="2028-01-19 21:26:23.939400011 +0000 UTC" Jul 15 16:56:26.425995 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.425943 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13276h29m57.513461054s" Jul 15 16:56:26.436507 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:26.436475 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fd95390_8b40_475c_ab93_ab641b315e09.slice/crio-903fd65d5551663bc41e433c791d77b1d06f3b50910dcfb9d61104cc12365fef WatchSource:0}: Error finding container 903fd65d5551663bc41e433c791d77b1d06f3b50910dcfb9d61104cc12365fef: Status 404 returned error can't find the container with id 903fd65d5551663bc41e433c791d77b1d06f3b50910dcfb9d61104cc12365fef Jul 15 16:56:26.563099 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.562580 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:26.563099 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:26.562720 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:26.568852 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.568751 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" event={"ID":"d626712c-c088-4ffe-b923-0b440dcc79d1","Type":"ContainerStarted","Data":"60b4acb02f26f859ef43476d0992bedb5bfe10b4faea5999a377ddcf9f4ad2b8"} Jul 15 16:56:26.578713 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.578640 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tzkgv" event={"ID":"cc6a2e63-80e9-44d9-9708-dc85bd571957","Type":"ContainerStarted","Data":"2e278d79d2a3ca75a61769a92e0276334f27d73d000ee889343c60fb2de4c2c7"} Jul 15 16:56:26.582833 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.582806 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6cqnq" event={"ID":"375d7469-6ffd-4be1-94bf-2782015de14d","Type":"ContainerStarted","Data":"fb99c92895aeaa6ca3e7eb0a8ba4b989d4e5d7271d753107fde99ec293e35789"} Jul 15 16:56:26.586703 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.586670 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zmjhz" event={"ID":"59101d7a-bd9b-4db5-b05b-4466aa58c6c5","Type":"ContainerStarted","Data":"6968c14cce1db6084676b350d9ee78dd57be39985c7637da11423b0f2cfcd97d"} Jul 15 16:56:26.588938 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.588915 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" event={"ID":"aac4144b-3d96-41bc-b02f-d2f5c39acc81","Type":"ContainerStarted","Data":"fec84568c73a1c0ea16cd9decd8ec60e48da059347e1e224e3e67accc7f2b820"} Jul 15 16:56:26.593464 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.593439 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8w7fh" event={"ID":"1fd95390-8b40-475c-ab93-ab641b315e09","Type":"ContainerStarted","Data":"903fd65d5551663bc41e433c791d77b1d06f3b50910dcfb9d61104cc12365fef"} Jul 15 16:56:26.603385 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.603359 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dx7t2" event={"ID":"76a7ad68-9351-4709-b119-ac5a58ca509e","Type":"ContainerStarted","Data":"854ca8378b2284cc025d9cb735a35070ce03f161af7cb93840c935652090c2d2"} Jul 15 16:56:26.608665 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.608594 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" event={"ID":"f3c2588fa3cc1960d37ff6f4e9a4674e","Type":"ContainerStarted","Data":"02986b1a9d19c3f3d4d4e54be461b8f06d9985c781d46b1409cb6fa9dc13f0dc"} Jul 15 16:56:26.615009 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.614955 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" event={"ID":"9863c9ecbe102ca15435dceaa71ff34e","Type":"ContainerStarted","Data":"03afd28c67a60a24c7d167597251c521de4c60711bec585b5f8af0c11615d399"} Jul 15 16:56:26.618809 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.618691 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" event={"ID":"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2","Type":"ContainerStarted","Data":"b0bfaa43a5adbe38c56371630611fd9a83cd1ba3b911886db444677b49cb1378"} Jul 15 16:56:26.978134 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:26.977582 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs\") pod \"network-metrics-daemon-mbqkc\" (UID: \"4f43f08e-aa70-42d2-9207-93156d956d7f\") " pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:26.978134 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:26.977742 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:26.978134 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:26.977807 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs podName:4f43f08e-aa70-42d2-9207-93156d956d7f nodeName:}" failed. No retries permitted until 2026-07-15 16:56:28.977788006 +0000 UTC m=+5.026785179 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs") pod "network-metrics-daemon-mbqkc" (UID: "4f43f08e-aa70-42d2-9207-93156d956d7f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:27.078843 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:27.078063 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2kmc\" (UniqueName: \"kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc\") pod \"network-check-target-btjr2\" (UID: \"8ba9f582-94a1-4bdf-b4de-6047b45368d2\") " pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:27.078843 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:27.078148 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret\") pod \"global-pull-secret-syncer-8xlk5\" (UID: \"1356768b-d3b0-4d99-98c7-42a27e4d0818\") " pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:27.078843 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:27.078266 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:27.078843 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:27.078340 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret podName:1356768b-d3b0-4d99-98c7-42a27e4d0818 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:29.078322223 +0000 UTC m=+5.127319396 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret") pod "global-pull-secret-syncer-8xlk5" (UID: "1356768b-d3b0-4d99-98c7-42a27e4d0818") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:27.078843 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:27.078739 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:56:27.078843 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:27.078757 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:56:27.078843 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:27.078769 2562 projected.go:196] Error preparing data for projected volume kube-api-access-q2kmc for pod openshift-network-diagnostics/network-check-target-btjr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:27.078843 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:27.078812 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc podName:8ba9f582-94a1-4bdf-b4de-6047b45368d2 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:29.078796959 +0000 UTC m=+5.127794129 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-q2kmc" (UniqueName: "kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc") pod "network-check-target-btjr2" (UID: "8ba9f582-94a1-4bdf-b4de-6047b45368d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:27.426748 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:27.426642 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 16:51:25 +0000 UTC" deadline="2028-03-31 13:52:32.753271918 +0000 UTC" Jul 15 16:56:27.426748 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:27.426694 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14996h56m5.326581055s" Jul 15 16:56:27.560735 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:27.560705 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:27.560901 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:27.560834 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:27.561165 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:27.560625 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:27.561274 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:27.561242 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:27.630366 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:27.630329 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v9jrp" event={"ID":"abe0c218-237b-4de9-88bb-9c2623db9bfa","Type":"ContainerStarted","Data":"987e42ab96a2c75126659b7c1880e3c142c2d3c0d4196d2cfcb69977e603c2bd"} Jul 15 16:56:27.722304 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:27.722005 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:56:28.564155 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:28.563673 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:28.564155 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:28.563805 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:28.994787 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:28.994324 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs\") pod \"network-metrics-daemon-mbqkc\" (UID: \"4f43f08e-aa70-42d2-9207-93156d956d7f\") " pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:28.994787 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:28.994484 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:28.994787 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:28.994538 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs podName:4f43f08e-aa70-42d2-9207-93156d956d7f nodeName:}" failed. No retries permitted until 2026-07-15 16:56:32.994525135 +0000 UTC m=+9.043522304 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs") pod "network-metrics-daemon-mbqkc" (UID: "4f43f08e-aa70-42d2-9207-93156d956d7f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:29.096247 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:29.095393 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2kmc\" (UniqueName: \"kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc\") pod \"network-check-target-btjr2\" (UID: \"8ba9f582-94a1-4bdf-b4de-6047b45368d2\") " pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:29.096247 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:29.095476 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret\") pod \"global-pull-secret-syncer-8xlk5\" (UID: \"1356768b-d3b0-4d99-98c7-42a27e4d0818\") " pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:29.096247 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:29.095621 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:29.096247 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:29.095683 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret podName:1356768b-d3b0-4d99-98c7-42a27e4d0818 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:33.095665351 +0000 UTC m=+9.144662544 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret") pod "global-pull-secret-syncer-8xlk5" (UID: "1356768b-d3b0-4d99-98c7-42a27e4d0818") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:29.096247 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:29.096067 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:56:29.096247 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:29.096085 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:56:29.096247 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:29.096097 2562 projected.go:196] Error preparing data for projected volume kube-api-access-q2kmc for pod openshift-network-diagnostics/network-check-target-btjr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:29.096247 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:29.096142 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc podName:8ba9f582-94a1-4bdf-b4de-6047b45368d2 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:33.096127723 +0000 UTC m=+9.145124897 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-q2kmc" (UniqueName: "kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc") pod "network-check-target-btjr2" (UID: "8ba9f582-94a1-4bdf-b4de-6047b45368d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:29.561271 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:29.561190 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:29.561429 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:29.561329 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:29.561429 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:29.561388 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:29.561560 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:29.561452 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:30.561430 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:30.561393 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:30.561898 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:30.561537 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:31.560682 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:31.560622 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:31.560853 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:31.560761 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:31.560912 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:31.560849 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:31.560976 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:31.560951 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:32.562154 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:32.561632 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:32.562154 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:32.561759 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:33.028227 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:33.028141 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs\") pod \"network-metrics-daemon-mbqkc\" (UID: \"4f43f08e-aa70-42d2-9207-93156d956d7f\") " pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:33.028394 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:33.028326 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:33.028394 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:33.028394 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs podName:4f43f08e-aa70-42d2-9207-93156d956d7f nodeName:}" failed. No retries permitted until 2026-07-15 16:56:41.028372956 +0000 UTC m=+17.077370139 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs") pod "network-metrics-daemon-mbqkc" (UID: "4f43f08e-aa70-42d2-9207-93156d956d7f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:33.128919 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:33.128827 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2kmc\" (UniqueName: \"kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc\") pod \"network-check-target-btjr2\" (UID: \"8ba9f582-94a1-4bdf-b4de-6047b45368d2\") " pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:33.128919 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:33.128905 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret\") pod \"global-pull-secret-syncer-8xlk5\" (UID: \"1356768b-d3b0-4d99-98c7-42a27e4d0818\") " pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:33.129312 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:33.129023 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:33.129312 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:33.129083 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret podName:1356768b-d3b0-4d99-98c7-42a27e4d0818 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:41.129064149 +0000 UTC m=+17.178061321 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret") pod "global-pull-secret-syncer-8xlk5" (UID: "1356768b-d3b0-4d99-98c7-42a27e4d0818") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:33.129622 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:33.129508 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:56:33.129622 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:33.129536 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:56:33.129622 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:33.129549 2562 projected.go:196] Error preparing data for projected volume kube-api-access-q2kmc for pod openshift-network-diagnostics/network-check-target-btjr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:33.129622 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:33.129597 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc podName:8ba9f582-94a1-4bdf-b4de-6047b45368d2 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:41.129581884 +0000 UTC m=+17.178579066 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-q2kmc" (UniqueName: "kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc") pod "network-check-target-btjr2" (UID: "8ba9f582-94a1-4bdf-b4de-6047b45368d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:33.560833 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:33.560710 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:33.560833 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:33.560758 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:33.561070 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:33.560851 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:33.561070 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:33.560992 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:34.562102 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:34.562072 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:34.562599 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:34.562194 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:35.561376 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:35.561279 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:35.561376 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:35.561335 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:35.561585 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:35.561433 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:35.561585 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:35.561550 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:36.561037 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:36.561005 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:36.561420 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:36.561126 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:37.561049 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:37.560972 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:37.561503 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:37.560982 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:37.561503 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:37.561099 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:37.561503 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:37.561195 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:38.561244 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:38.561209 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:38.561681 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:38.561345 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:39.560911 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:39.560885 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:39.561070 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:39.560888 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:39.561070 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:39.560989 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:39.561171 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:39.561077 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:40.561123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:40.561088 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:40.561526 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:40.561214 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:41.083899 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:41.083870 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs\") pod \"network-metrics-daemon-mbqkc\" (UID: \"4f43f08e-aa70-42d2-9207-93156d956d7f\") " pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:41.084074 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:41.084030 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:41.084133 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:41.084089 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs podName:4f43f08e-aa70-42d2-9207-93156d956d7f nodeName:}" failed. No retries permitted until 2026-07-15 16:56:57.084074129 +0000 UTC m=+33.133071298 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs") pod "network-metrics-daemon-mbqkc" (UID: "4f43f08e-aa70-42d2-9207-93156d956d7f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:41.184702 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:41.184670 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2kmc\" (UniqueName: \"kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc\") pod \"network-check-target-btjr2\" (UID: \"8ba9f582-94a1-4bdf-b4de-6047b45368d2\") " pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:41.184892 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:41.184745 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret\") pod \"global-pull-secret-syncer-8xlk5\" (UID: \"1356768b-d3b0-4d99-98c7-42a27e4d0818\") " pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:41.184892 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:41.184810 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:56:41.184892 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:41.184834 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:56:41.184892 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:41.184846 2562 projected.go:196] Error preparing data for projected volume kube-api-access-q2kmc for pod openshift-network-diagnostics/network-check-target-btjr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:41.184892 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:41.184886 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:41.185114 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:41.184912 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc podName:8ba9f582-94a1-4bdf-b4de-6047b45368d2 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:57.184893234 +0000 UTC m=+33.233890410 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-q2kmc" (UniqueName: "kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc") pod "network-check-target-btjr2" (UID: "8ba9f582-94a1-4bdf-b4de-6047b45368d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:41.185114 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:41.184938 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret podName:1356768b-d3b0-4d99-98c7-42a27e4d0818 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:57.184921503 +0000 UTC m=+33.233918672 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret") pod "global-pull-secret-syncer-8xlk5" (UID: "1356768b-d3b0-4d99-98c7-42a27e4d0818") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:41.560992 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:41.560964 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:41.561178 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:41.561000 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:41.576660 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:41.576630 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:41.576796 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:41.576772 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:42.560875 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:42.560839 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:42.561058 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:42.560987 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:43.560981 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:43.560942 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:43.561436 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:43.560990 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:43.561436 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:43.561093 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:43.561436 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:43.561246 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:44.561883 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:44.561851 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:44.562323 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:44.561963 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:45.560662 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:45.560527 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:45.560750 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:45.560574 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:45.560787 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:45.560744 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:45.560827 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:45.560807 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:45.666580 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:45.666525 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" event={"ID":"d626712c-c088-4ffe-b923-0b440dcc79d1","Type":"ContainerStarted","Data":"3b0ebe95052967fc1ac7522dbf32e84758c7e789d80a25d2256535b7358b07ff"} Jul 15 16:56:45.667922 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:45.667900 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6cqnq" event={"ID":"375d7469-6ffd-4be1-94bf-2782015de14d","Type":"ContainerStarted","Data":"4b02382b389d8495a54f53ef0dd5039786130d9ea2bf2640a0d3c2a7003239f7"} Jul 15 16:56:45.669216 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:45.669180 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" event={"ID":"9863c9ecbe102ca15435dceaa71ff34e","Type":"ContainerStarted","Data":"3020343bd364a1d501fe9c6b7831a6fbba50e5381bd686fff192ff903f055fc8"} Jul 15 16:56:45.675847 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:45.675826 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" event={"ID":"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2","Type":"ContainerStarted","Data":"3406f289e16c947f3baf91bbe44e06c9a72bde38a9ab33d1496c9ffb60246ede"} Jul 15 16:56:45.675909 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:45.675899 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" event={"ID":"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2","Type":"ContainerStarted","Data":"57e8fe1cc92cd720f6842906eb2037833925ffac6094eb53b35b6bdb496b3555"} Jul 15 16:56:45.685446 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:45.685409 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xn7j8" podStartSLOduration=2.761872498 podStartE2EDuration="21.685398091s" totalImagesPullingTime="18.923525593s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:45.685151621 +0000 UTC m=+21.734148812" watchObservedRunningTime="2026-07-15 16:56:45.685398091 +0000 UTC m=+21.734395281" Jul 15 16:56:45.701545 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:45.701505 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6cqnq" podStartSLOduration=2.51337935 podStartE2EDuration="21.701493617s" totalImagesPullingTime="19.188114267s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:45.701236051 +0000 UTC m=+21.750233241" watchObservedRunningTime="2026-07-15 16:56:45.701493617 +0000 UTC m=+21.750490807" Jul 15 16:56:45.714615 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:45.714584 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-118.ec2.internal" podStartSLOduration=20.714573136 podStartE2EDuration="20.714573136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:45.714491266 +0000 UTC m=+21.763488456" watchObservedRunningTime="2026-07-15 16:56:45.714573136 +0000 UTC m=+21.763570323" Jul 15 16:56:46.562517 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.562349 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:46.562671 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:46.562616 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:46.679283 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.679181 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tzkgv" event={"ID":"cc6a2e63-80e9-44d9-9708-dc85bd571957","Type":"ContainerStarted","Data":"a9946106103ccd29e1e0a97960120ae1da52e08220cb47bce6b14cb4cacb5c43"} Jul 15 16:56:46.680904 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.680876 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zmjhz" event={"ID":"59101d7a-bd9b-4db5-b05b-4466aa58c6c5","Type":"ContainerStarted","Data":"b893933c27d6930add1e6446ef66afd0a16983384a26433e76e00876decc609b"} Jul 15 16:56:46.682483 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.682455 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" event={"ID":"aac4144b-3d96-41bc-b02f-d2f5c39acc81","Type":"ContainerStarted","Data":"ba71b4e80455350c2bf853072ac84f2c01a50d92c66216069e67a636d2ed355b"} Jul 15 16:56:46.684054 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.684030 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8w7fh" event={"ID":"1fd95390-8b40-475c-ab93-ab641b315e09","Type":"ContainerStarted","Data":"bb2fea84f71adeba8c6e84be9298a273da9a753240e866e125ca1ea4f8393591"} Jul 15 16:56:46.685651 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.685631 2562 generic.go:356] "Generic (PLEG): container finished" podID="76a7ad68-9351-4709-b119-ac5a58ca509e" containerID="bfeea72a044051bea9a7c33b0904a60d33044e40ac79ccb4e207ba4227581488" exitCode=0 Jul 15 16:56:46.685746 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.685703 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dx7t2" event={"ID":"76a7ad68-9351-4709-b119-ac5a58ca509e","Type":"ContainerDied","Data":"bfeea72a044051bea9a7c33b0904a60d33044e40ac79ccb4e207ba4227581488"} Jul 15 16:56:46.687435 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.687403 2562 generic.go:356] "Generic (PLEG): container finished" podID="f3c2588fa3cc1960d37ff6f4e9a4674e" containerID="806a3a85ea40f4c8a499bc8db911f0ed68cb71ca021608ceaaea15b14fff3bb7" exitCode=0 Jul 15 16:56:46.687512 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.687483 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" event={"ID":"f3c2588fa3cc1960d37ff6f4e9a4674e","Type":"ContainerDied","Data":"806a3a85ea40f4c8a499bc8db911f0ed68cb71ca021608ceaaea15b14fff3bb7"} Jul 15 16:56:46.690562 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.690543 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 16:56:46.690910 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.690886 2562 generic.go:356] "Generic (PLEG): container finished" podID="e8547271-3ba8-4a2c-8e1e-d3c18de88bc2" containerID="3406f289e16c947f3baf91bbe44e06c9a72bde38a9ab33d1496c9ffb60246ede" exitCode=1 Jul 15 16:56:46.690973 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.690947 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" event={"ID":"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2","Type":"ContainerDied","Data":"3406f289e16c947f3baf91bbe44e06c9a72bde38a9ab33d1496c9ffb60246ede"} Jul 15 16:56:46.691036 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.691021 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" event={"ID":"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2","Type":"ContainerStarted","Data":"49e3edc0707e163c6a8c76923238a8c8dcef057d34f28872ea712f7e6daa02ac"} Jul 15 16:56:46.691089 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.691078 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" event={"ID":"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2","Type":"ContainerStarted","Data":"720bf461e0327f54cf2c221275f40b0ae02f0ee3876a3e797e3aaf11c831e320"} Jul 15 16:56:46.691132 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.691122 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" event={"ID":"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2","Type":"ContainerStarted","Data":"3d4e1b95c17e3ae60da098855c02b8d3973aa35112ef4fb14577eb20d656ef8a"} Jul 15 16:56:46.691179 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.691162 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" event={"ID":"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2","Type":"ContainerStarted","Data":"538e0455fb5229f92a3bd2ab086b203fce4c472d9e300da579f3cd7adbf0b341"} Jul 15 16:56:46.692449 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.692310 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v9jrp" event={"ID":"abe0c218-237b-4de9-88bb-9c2623db9bfa","Type":"ContainerStarted","Data":"fe2d32c75e2c8773c46b65294b9ee895ced453369397b7adacd93958f26b9549"} Jul 15 16:56:46.697753 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.697718 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tzkgv" podStartSLOduration=3.945281198 podStartE2EDuration="22.697707121s" totalImagesPullingTime="18.752425923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:46.697151834 +0000 UTC m=+22.746149028" watchObservedRunningTime="2026-07-15 16:56:46.697707121 +0000 UTC m=+22.746704312" Jul 15 16:56:46.779949 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.779851 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8w7fh" podStartSLOduration=4.323415832 podStartE2EDuration="22.779837458s" totalImagesPullingTime="18.456421626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:46.779224926 +0000 UTC m=+22.828222117" watchObservedRunningTime="2026-07-15 16:56:46.779837458 +0000 UTC m=+22.828834649" Jul 15 16:56:46.794210 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.794164 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-v9jrp" podStartSLOduration=5.037959982 podStartE2EDuration="22.794149828s" totalImagesPullingTime="17.756189846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:46.793722018 +0000 UTC m=+22.842719209" watchObservedRunningTime="2026-07-15 16:56:46.794149828 +0000 UTC m=+22.843147018" Jul 15 16:56:46.811390 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:46.811355 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zmjhz" podStartSLOduration=3.651707769 podStartE2EDuration="22.811342856s" totalImagesPullingTime="19.159635087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:46.811233332 +0000 UTC m=+22.860230523" watchObservedRunningTime="2026-07-15 16:56:46.811342856 +0000 UTC m=+22.860340090" Jul 15 16:56:47.228741 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:47.228720 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 16:56:47.436938 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:47.436799 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T16:56:47.22873731Z","UUID":"df6db141-16dd-47dc-a88a-70fa90488789","Handler":null,"Name":"","Endpoint":""} Jul 15 16:56:47.438528 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:47.438507 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 16:56:47.438656 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:47.438546 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 16:56:47.560604 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:47.560571 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:47.560770 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:47.560694 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:47.560770 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:47.560749 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:47.560877 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:47.560857 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:47.696647 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:47.696567 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" event={"ID":"aac4144b-3d96-41bc-b02f-d2f5c39acc81","Type":"ContainerStarted","Data":"4b497151bbcd742a5f6d1f08c2531874faebdcd830c75b369d93772b9e51b326"} Jul 15 16:56:47.698493 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:47.698467 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" event={"ID":"f3c2588fa3cc1960d37ff6f4e9a4674e","Type":"ContainerStarted","Data":"547c041f0ee7c048be0e2ad7cdf7dea092b108dbad474faaececff7082bf1724"} Jul 15 16:56:47.714354 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:47.714308 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-118.ec2.internal" podStartSLOduration=22.714281999 podStartE2EDuration="22.714281999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:47.714091176 +0000 UTC m=+23.763088366" watchObservedRunningTime="2026-07-15 16:56:47.714281999 +0000 UTC m=+23.763279189" Jul 15 16:56:48.560462 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:48.560391 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:48.560570 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:48.560519 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:48.704591 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:48.704561 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 16:56:48.705076 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:48.705051 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" event={"ID":"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2","Type":"ContainerStarted","Data":"650da90904c853ed7fb7d07174739508cab5fe09a29d2f4ca4f9c630ecc633f5"} Jul 15 16:56:48.707495 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:48.707469 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" event={"ID":"aac4144b-3d96-41bc-b02f-d2f5c39acc81","Type":"ContainerStarted","Data":"2511851312f68754d4c35c163b1a58bc3a579322086adb57b62f83dfc6c6b99c"} Jul 15 16:56:49.055255 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:49.055223 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-v9jrp" Jul 15 16:56:49.055838 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:49.055816 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-v9jrp" Jul 15 16:56:49.071868 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:49.071824 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x7bdh" podStartSLOduration=3.646010209 podStartE2EDuration="25.071809316s" totalImagesPullingTime="21.425799107s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:48.726744497 +0000 UTC m=+24.775741688" watchObservedRunningTime="2026-07-15 16:56:49.071809316 +0000 UTC m=+25.120806507" Jul 15 16:56:49.561387 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:49.561356 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:49.561581 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:49.561356 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:49.561581 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:49.561481 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:49.561581 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:49.561550 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:49.709750 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:49.709719 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-v9jrp" Jul 15 16:56:49.710262 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:49.710238 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-v9jrp" Jul 15 16:56:50.560804 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:50.560770 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:50.560949 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:50.560901 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:51.560748 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:51.560581 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:51.561358 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:51.560610 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:51.561358 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:51.560838 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:51.561358 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:51.560876 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:51.714669 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:51.714640 2562 generic.go:356] "Generic (PLEG): container finished" podID="76a7ad68-9351-4709-b119-ac5a58ca509e" containerID="80340b24477914809b98b0b7837c9314f7520d63258d01286297d0003904d785" exitCode=0 Jul 15 16:56:51.714865 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:51.714724 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dx7t2" event={"ID":"76a7ad68-9351-4709-b119-ac5a58ca509e","Type":"ContainerDied","Data":"80340b24477914809b98b0b7837c9314f7520d63258d01286297d0003904d785"} Jul 15 16:56:51.717926 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:51.717911 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 16:56:51.718351 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:51.718330 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" event={"ID":"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2","Type":"ContainerStarted","Data":"ee63d47a6801abe023429fd8e65f032845dfce529a4fde13b8516313dce1f6c3"} Jul 15 16:56:51.718728 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:51.718714 2562 scope.go:117] "RemoveContainer" containerID="3406f289e16c947f3baf91bbe44e06c9a72bde38a9ab33d1496c9ffb60246ede" Jul 15 16:56:52.561219 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.561002 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:52.561589 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:52.561257 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:52.722875 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.722843 2562 generic.go:356] "Generic (PLEG): container finished" podID="76a7ad68-9351-4709-b119-ac5a58ca509e" containerID="cb8376472ebfa51e3ca6b784b9e4cd798f10dd6a62a48412ed9c09971bd48b6e" exitCode=0 Jul 15 16:56:52.723046 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.722918 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dx7t2" event={"ID":"76a7ad68-9351-4709-b119-ac5a58ca509e","Type":"ContainerDied","Data":"cb8376472ebfa51e3ca6b784b9e4cd798f10dd6a62a48412ed9c09971bd48b6e"} Jul 15 16:56:52.731032 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.731013 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 16:56:52.731405 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.731382 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" event={"ID":"e8547271-3ba8-4a2c-8e1e-d3c18de88bc2","Type":"ContainerStarted","Data":"df9363b63608db1472d518475a3e100f688c80a67f1adb2ac33a6e3f4ca75eb6"} Jul 15 16:56:52.731740 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.731720 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:52.731851 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.731784 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:52.731851 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.731830 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:52.748533 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.748512 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:52.748982 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.748969 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:56:52.772021 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.771984 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" podStartSLOduration=9.425282841 podStartE2EDuration="28.771970823s" totalImagesPullingTime="19.346687982s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:52.771345327 +0000 UTC m=+28.820342518" watchObservedRunningTime="2026-07-15 16:56:52.771970823 +0000 UTC m=+28.820968012" Jul 15 16:56:52.960846 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.960818 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mbqkc"] Jul 15 16:56:52.960999 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.960938 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:52.961064 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:52.961044 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:52.963732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.963707 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-btjr2"] Jul 15 16:56:52.963856 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.963800 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:52.963913 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:52.963872 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:52.964424 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.964404 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8xlk5"] Jul 15 16:56:52.964512 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:52.964499 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:52.964611 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:52.964591 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:54.562406 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:54.562373 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:54.562786 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:54.562506 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:54.562956 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:54.562934 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:54.563081 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:54.563060 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:54.563156 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:54.563126 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:54.563250 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:54.563234 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:54.738792 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:54.738699 2562 generic.go:356] "Generic (PLEG): container finished" podID="76a7ad68-9351-4709-b119-ac5a58ca509e" containerID="70ca62f05a0cc0d8b45a65a72245fee406e9544cdc6bddba04a36b2d9fd567be" exitCode=0 Jul 15 16:56:54.738946 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:54.738782 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dx7t2" event={"ID":"76a7ad68-9351-4709-b119-ac5a58ca509e","Type":"ContainerDied","Data":"70ca62f05a0cc0d8b45a65a72245fee406e9544cdc6bddba04a36b2d9fd567be"} Jul 15 16:56:56.564303 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:56.564120 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:56.564705 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:56.564155 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:56.564705 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:56.564432 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:56:56.564705 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:56.564189 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:56.564705 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:56.564475 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xlk5" podUID="1356768b-d3b0-4d99-98c7-42a27e4d0818" Jul 15 16:56:56.564705 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:56.564537 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-btjr2" podUID="8ba9f582-94a1-4bdf-b4de-6047b45368d2" Jul 15 16:56:57.098056 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:57.098025 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs\") pod \"network-metrics-daemon-mbqkc\" (UID: \"4f43f08e-aa70-42d2-9207-93156d956d7f\") " pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:57.098271 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:57.098148 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:57.098271 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:57.098200 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs podName:4f43f08e-aa70-42d2-9207-93156d956d7f nodeName:}" failed. No retries permitted until 2026-07-15 16:57:29.098186657 +0000 UTC m=+65.147183829 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs") pod "network-metrics-daemon-mbqkc" (UID: "4f43f08e-aa70-42d2-9207-93156d956d7f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:56:57.199238 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:57.199208 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret\") pod \"global-pull-secret-syncer-8xlk5\" (UID: \"1356768b-d3b0-4d99-98c7-42a27e4d0818\") " pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:57.199406 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:57.199267 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2kmc\" (UniqueName: \"kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc\") pod \"network-check-target-btjr2\" (UID: \"8ba9f582-94a1-4bdf-b4de-6047b45368d2\") " pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:57.199406 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:57.199387 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:57.199406 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:57.199399 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:56:57.199558 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:57.199416 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:56:57.199558 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:57.199425 2562 projected.go:196] Error preparing data for projected volume kube-api-access-q2kmc for pod openshift-network-diagnostics/network-check-target-btjr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:57.199558 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:57.199462 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret podName:1356768b-d3b0-4d99-98c7-42a27e4d0818 nodeName:}" failed. No retries permitted until 2026-07-15 16:57:29.199440484 +0000 UTC m=+65.248437672 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret") pod "global-pull-secret-syncer-8xlk5" (UID: "1356768b-d3b0-4d99-98c7-42a27e4d0818") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:56:57.199558 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:57.199479 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc podName:8ba9f582-94a1-4bdf-b4de-6047b45368d2 nodeName:}" failed. No retries permitted until 2026-07-15 16:57:29.199472253 +0000 UTC m=+65.248469423 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-q2kmc" (UniqueName: "kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc") pod "network-check-target-btjr2" (UID: "8ba9f582-94a1-4bdf-b4de-6047b45368d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:56:58.310516 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.310484 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-118.ec2.internal" event="NodeReady" Jul 15 16:56:58.310990 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.310635 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 16:56:58.359866 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.359796 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dph8q"] Jul 15 16:56:58.392194 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.392164 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jd6qq"] Jul 15 16:56:58.392370 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.392196 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dph8q" Jul 15 16:56:58.395009 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.394988 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 16:56:58.395112 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.395040 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 16:56:58.395112 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.395057 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lllhc\"" Jul 15 16:56:58.406064 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.406043 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dph8q"] Jul 15 16:56:58.406162 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.406079 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jd6qq"] Jul 15 16:56:58.406162 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.406156 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jd6qq" Jul 15 16:56:58.408595 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.408579 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rf5xj\"" Jul 15 16:56:58.409032 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.408818 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 16:56:58.409032 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.408887 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 16:56:58.409032 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.408890 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 16:56:58.508123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.508093 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqdc6\" (UniqueName: \"kubernetes.io/projected/65c4ae32-87b7-46f2-8370-510449c1020e-kube-api-access-wqdc6\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:56:58.508123 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.508127 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65c4ae32-87b7-46f2-8370-510449c1020e-tmp-dir\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:56:58.508286 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.508174 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:56:58.508286 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.508195 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/65c4ae32-87b7-46f2-8370-510449c1020e-config-volume\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:56:58.508286 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.508212 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzz2q\" (UniqueName: \"kubernetes.io/projected/f5d75ab3-c94f-4349-953a-05abad93f858-kube-api-access-kzz2q\") pod \"ingress-canary-jd6qq\" (UID: \"f5d75ab3-c94f-4349-953a-05abad93f858\") " pod="openshift-ingress-canary/ingress-canary-jd6qq" Jul 15 16:56:58.508286 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.508272 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5d75ab3-c94f-4349-953a-05abad93f858-cert\") pod \"ingress-canary-jd6qq\" (UID: \"f5d75ab3-c94f-4349-953a-05abad93f858\") " pod="openshift-ingress-canary/ingress-canary-jd6qq" Jul 15 16:56:58.561399 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.561369 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:56:58.561399 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.561379 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:56:58.561591 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.561575 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:56:58.564093 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.564074 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 16:56:58.564410 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.564389 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 16:56:58.564410 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.564407 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pqk7d\"" Jul 15 16:56:58.564574 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.564459 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7h7tx\"" Jul 15 16:56:58.564574 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.564393 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 16:56:58.564574 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.564390 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 16:56:58.608545 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.608525 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5d75ab3-c94f-4349-953a-05abad93f858-cert\") pod \"ingress-canary-jd6qq\" (UID: \"f5d75ab3-c94f-4349-953a-05abad93f858\") " pod="openshift-ingress-canary/ingress-canary-jd6qq" Jul 15 16:56:58.608641 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.608551 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqdc6\" (UniqueName: \"kubernetes.io/projected/65c4ae32-87b7-46f2-8370-510449c1020e-kube-api-access-wqdc6\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:56:58.608641 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.608569 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65c4ae32-87b7-46f2-8370-510449c1020e-tmp-dir\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:56:58.608753 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.608733 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:56:58.608808 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.608779 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/65c4ae32-87b7-46f2-8370-510449c1020e-config-volume\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:56:58.608859 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.608815 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65c4ae32-87b7-46f2-8370-510449c1020e-tmp-dir\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:56:58.608859 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.608825 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzz2q\" (UniqueName: \"kubernetes.io/projected/f5d75ab3-c94f-4349-953a-05abad93f858-kube-api-access-kzz2q\") pod \"ingress-canary-jd6qq\" (UID: \"f5d75ab3-c94f-4349-953a-05abad93f858\") " pod="openshift-ingress-canary/ingress-canary-jd6qq" Jul 15 16:56:58.608951 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:58.608866 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:56:58.608951 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:58.608947 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls podName:65c4ae32-87b7-46f2-8370-510449c1020e nodeName:}" failed. No retries permitted until 2026-07-15 16:56:59.108931183 +0000 UTC m=+35.157928372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls") pod "dns-default-dph8q" (UID: "65c4ae32-87b7-46f2-8370-510449c1020e") : secret "dns-default-metrics-tls" not found Jul 15 16:56:58.609240 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.609225 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/65c4ae32-87b7-46f2-8370-510449c1020e-config-volume\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:56:58.612551 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.612510 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5d75ab3-c94f-4349-953a-05abad93f858-cert\") pod \"ingress-canary-jd6qq\" (UID: \"f5d75ab3-c94f-4349-953a-05abad93f858\") " pod="openshift-ingress-canary/ingress-canary-jd6qq" Jul 15 16:56:58.617505 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.617484 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqdc6\" (UniqueName: \"kubernetes.io/projected/65c4ae32-87b7-46f2-8370-510449c1020e-kube-api-access-wqdc6\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:56:58.617877 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.617861 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzz2q\" (UniqueName: \"kubernetes.io/projected/f5d75ab3-c94f-4349-953a-05abad93f858-kube-api-access-kzz2q\") pod \"ingress-canary-jd6qq\" (UID: \"f5d75ab3-c94f-4349-953a-05abad93f858\") " pod="openshift-ingress-canary/ingress-canary-jd6qq" Jul 15 16:56:58.714622 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.714596 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jd6qq" Jul 15 16:56:58.877358 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:58.877160 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jd6qq"] Jul 15 16:56:58.880281 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:56:58.880250 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5d75ab3_c94f_4349_953a_05abad93f858.slice/crio-d5d7a165c9ccb2d6e35e275086b89b348d4c2ec13fef3464769123e6e63c9c35 WatchSource:0}: Error finding container d5d7a165c9ccb2d6e35e275086b89b348d4c2ec13fef3464769123e6e63c9c35: Status 404 returned error can't find the container with id d5d7a165c9ccb2d6e35e275086b89b348d4c2ec13fef3464769123e6e63c9c35 Jul 15 16:56:59.112911 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:59.112877 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:56:59.113059 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:59.113021 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:56:59.113104 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:56:59.113082 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls podName:65c4ae32-87b7-46f2-8370-510449c1020e nodeName:}" failed. No retries permitted until 2026-07-15 16:57:00.113067788 +0000 UTC m=+36.162064957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls") pod "dns-default-dph8q" (UID: "65c4ae32-87b7-46f2-8370-510449c1020e") : secret "dns-default-metrics-tls" not found Jul 15 16:56:59.752137 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:59.752098 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jd6qq" event={"ID":"f5d75ab3-c94f-4349-953a-05abad93f858","Type":"ContainerStarted","Data":"d5d7a165c9ccb2d6e35e275086b89b348d4c2ec13fef3464769123e6e63c9c35"} Jul 15 16:56:59.755114 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:59.755089 2562 generic.go:356] "Generic (PLEG): container finished" podID="76a7ad68-9351-4709-b119-ac5a58ca509e" containerID="7aa13de7036b433fe159647111b304a7ccf6df799468c3045adfb2a86adb4e1a" exitCode=0 Jul 15 16:56:59.755236 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:56:59.755138 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dx7t2" event={"ID":"76a7ad68-9351-4709-b119-ac5a58ca509e","Type":"ContainerDied","Data":"7aa13de7036b433fe159647111b304a7ccf6df799468c3045adfb2a86adb4e1a"} Jul 15 16:57:00.121625 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:00.121536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:57:00.121788 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:57:00.121653 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:57:00.121788 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:57:00.121711 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls podName:65c4ae32-87b7-46f2-8370-510449c1020e nodeName:}" failed. No retries permitted until 2026-07-15 16:57:02.121692726 +0000 UTC m=+38.170689897 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls") pod "dns-default-dph8q" (UID: "65c4ae32-87b7-46f2-8370-510449c1020e") : secret "dns-default-metrics-tls" not found Jul 15 16:57:00.760953 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:00.760917 2562 generic.go:356] "Generic (PLEG): container finished" podID="76a7ad68-9351-4709-b119-ac5a58ca509e" containerID="7995e481259e0b5e20a460669ee87c07b1a498f35fe38c05dfa3db25f69d0a4e" exitCode=0 Jul 15 16:57:00.761373 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:00.760969 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dx7t2" event={"ID":"76a7ad68-9351-4709-b119-ac5a58ca509e","Type":"ContainerDied","Data":"7995e481259e0b5e20a460669ee87c07b1a498f35fe38c05dfa3db25f69d0a4e"} Jul 15 16:57:01.764733 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:01.764693 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jd6qq" event={"ID":"f5d75ab3-c94f-4349-953a-05abad93f858","Type":"ContainerStarted","Data":"c3a8b59dd2ba4f44a6fcb4b89d74d227488f92ff8123f9fcbe071d4e9460561a"} Jul 15 16:57:01.768012 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:01.767950 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dx7t2" event={"ID":"76a7ad68-9351-4709-b119-ac5a58ca509e","Type":"ContainerStarted","Data":"c5d150be9240858c15a03a3352492686be5e585954492a8f8e7c9131812250d0"} Jul 15 16:57:01.784455 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:01.784414 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jd6qq" podStartSLOduration=1.183791598 podStartE2EDuration="3.784400617s" totalImagesPullingTime="2.600609019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:01.783537344 +0000 UTC m=+37.832534533" watchObservedRunningTime="2026-07-15 16:57:01.784400617 +0000 UTC m=+37.833397809" Jul 15 16:57:01.806441 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:01.806394 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dx7t2" podStartSLOduration=8.492400107 podStartE2EDuration="37.806380131s" totalImagesPullingTime="29.313980024s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:01.804901883 +0000 UTC m=+37.853899073" watchObservedRunningTime="2026-07-15 16:57:01.806380131 +0000 UTC m=+37.855377322" Jul 15 16:57:02.134619 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:02.134537 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:57:02.134743 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:57:02.134657 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:57:02.134743 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:57:02.134712 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls podName:65c4ae32-87b7-46f2-8370-510449c1020e nodeName:}" failed. No retries permitted until 2026-07-15 16:57:06.134696947 +0000 UTC m=+42.183694116 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls") pod "dns-default-dph8q" (UID: "65c4ae32-87b7-46f2-8370-510449c1020e") : secret "dns-default-metrics-tls" not found Jul 15 16:57:06.160796 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:06.160756 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:57:06.161155 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:57:06.160901 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:57:06.161155 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:57:06.160960 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls podName:65c4ae32-87b7-46f2-8370-510449c1020e nodeName:}" failed. No retries permitted until 2026-07-15 16:57:14.160944478 +0000 UTC m=+50.209941654 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls") pod "dns-default-dph8q" (UID: "65c4ae32-87b7-46f2-8370-510449c1020e") : secret "dns-default-metrics-tls" not found Jul 15 16:57:14.208423 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:14.208363 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:57:14.209013 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:57:14.208524 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:57:14.209013 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:57:14.208614 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls podName:65c4ae32-87b7-46f2-8370-510449c1020e nodeName:}" failed. No retries permitted until 2026-07-15 16:57:30.208594535 +0000 UTC m=+66.257591721 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls") pod "dns-default-dph8q" (UID: "65c4ae32-87b7-46f2-8370-510449c1020e") : secret "dns-default-metrics-tls" not found Jul 15 16:57:24.752913 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:24.752887 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wlpcj" Jul 15 16:57:29.103526 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:29.103472 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs\") pod \"network-metrics-daemon-mbqkc\" (UID: \"4f43f08e-aa70-42d2-9207-93156d956d7f\") " pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:57:29.106134 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:29.106114 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 16:57:29.113812 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:57:29.113794 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 15 16:57:29.113862 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:57:29.113848 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs podName:4f43f08e-aa70-42d2-9207-93156d956d7f nodeName:}" failed. No retries permitted until 2026-07-15 16:58:33.113832628 +0000 UTC m=+129.162829797 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs") pod "network-metrics-daemon-mbqkc" (UID: "4f43f08e-aa70-42d2-9207-93156d956d7f") : secret "metrics-daemon-secret" not found Jul 15 16:57:29.203941 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:29.203904 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret\") pod \"global-pull-secret-syncer-8xlk5\" (UID: \"1356768b-d3b0-4d99-98c7-42a27e4d0818\") " pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:57:29.204025 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:29.203962 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2kmc\" (UniqueName: \"kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc\") pod \"network-check-target-btjr2\" (UID: \"8ba9f582-94a1-4bdf-b4de-6047b45368d2\") " pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:57:29.206845 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:29.206822 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 16:57:29.206950 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:29.206900 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 16:57:29.216732 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:29.216715 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 16:57:29.218202 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:29.218185 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1356768b-d3b0-4d99-98c7-42a27e4d0818-original-pull-secret\") pod \"global-pull-secret-syncer-8xlk5\" (UID: \"1356768b-d3b0-4d99-98c7-42a27e4d0818\") " pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:57:29.227535 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:29.227516 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2kmc\" (UniqueName: \"kubernetes.io/projected/8ba9f582-94a1-4bdf-b4de-6047b45368d2-kube-api-access-q2kmc\") pod \"network-check-target-btjr2\" (UID: \"8ba9f582-94a1-4bdf-b4de-6047b45368d2\") " pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:57:29.481095 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:29.481060 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xlk5" Jul 15 16:57:29.483876 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:29.483854 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7h7tx\"" Jul 15 16:57:29.491969 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:29.491951 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:57:29.606569 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:29.606491 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8xlk5"] Jul 15 16:57:29.610245 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:57:29.610223 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1356768b_d3b0_4d99_98c7_42a27e4d0818.slice/crio-bffeb506f473bc3c190c7ff391c26deb619d5f37f4be01e1c4b596580a8d5583 WatchSource:0}: Error finding container bffeb506f473bc3c190c7ff391c26deb619d5f37f4be01e1c4b596580a8d5583: Status 404 returned error can't find the container with id bffeb506f473bc3c190c7ff391c26deb619d5f37f4be01e1c4b596580a8d5583 Jul 15 16:57:29.620386 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:29.620364 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-btjr2"] Jul 15 16:57:29.623420 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:57:29.623393 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba9f582_94a1_4bdf_b4de_6047b45368d2.slice/crio-f6be06ef871d23c07733a6379228d9bdb4ed4d310700b0f273c88df3f650f0a5 WatchSource:0}: Error finding container f6be06ef871d23c07733a6379228d9bdb4ed4d310700b0f273c88df3f650f0a5: Status 404 returned error can't find the container with id f6be06ef871d23c07733a6379228d9bdb4ed4d310700b0f273c88df3f650f0a5 Jul 15 16:57:29.831216 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:29.831117 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8xlk5" event={"ID":"1356768b-d3b0-4d99-98c7-42a27e4d0818","Type":"ContainerStarted","Data":"bffeb506f473bc3c190c7ff391c26deb619d5f37f4be01e1c4b596580a8d5583"} Jul 15 16:57:29.832179 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:29.832152 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-btjr2" event={"ID":"8ba9f582-94a1-4bdf-b4de-6047b45368d2","Type":"ContainerStarted","Data":"f6be06ef871d23c07733a6379228d9bdb4ed4d310700b0f273c88df3f650f0a5"} Jul 15 16:57:30.210422 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:30.210392 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:57:30.210793 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:57:30.210543 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:57:30.210793 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:57:30.210608 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls podName:65c4ae32-87b7-46f2-8370-510449c1020e nodeName:}" failed. No retries permitted until 2026-07-15 16:58:02.210591711 +0000 UTC m=+98.259588881 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls") pod "dns-default-dph8q" (UID: "65c4ae32-87b7-46f2-8370-510449c1020e") : secret "dns-default-metrics-tls" not found Jul 15 16:57:34.846762 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:34.846726 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-btjr2" event={"ID":"8ba9f582-94a1-4bdf-b4de-6047b45368d2","Type":"ContainerStarted","Data":"c0720e005573c3f5809184db26a5d87a124a2cc372ab24ad4ca7803f27aa098f"} Jul 15 16:57:34.847196 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:34.846836 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:57:34.848150 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:34.848130 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8xlk5" event={"ID":"1356768b-d3b0-4d99-98c7-42a27e4d0818","Type":"ContainerStarted","Data":"265c6512fc77a9848bad14e2b7cdc56d2677b6d5071dfda789839759760c901d"} Jul 15 16:57:34.866188 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:34.866141 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-btjr2" podStartSLOduration=66.540921041 podStartE2EDuration="1m10.866127234s" totalImagesPullingTime="4.325206193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:34.865410773 +0000 UTC m=+70.914408018" watchObservedRunningTime="2026-07-15 16:57:34.866127234 +0000 UTC m=+70.915124425" Jul 15 16:57:34.882337 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:34.882282 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8xlk5" podStartSLOduration=66.538592518 podStartE2EDuration="1m10.882271456s" totalImagesPullingTime="4.343678938s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:34.881670016 +0000 UTC m=+70.930667207" watchObservedRunningTime="2026-07-15 16:57:34.882271456 +0000 UTC m=+70.931268647" Jul 15 16:57:36.180621 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.180585 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54bb49d69d-864rn"] Jul 15 16:57:36.183375 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.183355 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54bb49d69d-864rn" Jul 15 16:57:36.185874 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.185853 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 15 16:57:36.187143 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.187124 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-bwwjh\"" Jul 15 16:57:36.187235 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.187174 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 15 16:57:36.187306 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.187235 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 15 16:57:36.187306 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.187246 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 15 16:57:36.193029 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.193008 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54bb49d69d-864rn"] Jul 15 16:57:36.357043 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.357009 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgc7j\" (UniqueName: \"kubernetes.io/projected/c3446ceb-be73-4e8a-8393-47816c6fc3ca-kube-api-access-hgc7j\") pod \"managed-serviceaccount-addon-agent-54bb49d69d-864rn\" (UID: \"c3446ceb-be73-4e8a-8393-47816c6fc3ca\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54bb49d69d-864rn" Jul 15 16:57:36.357213 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.357069 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3446ceb-be73-4e8a-8393-47816c6fc3ca-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-54bb49d69d-864rn\" (UID: \"c3446ceb-be73-4e8a-8393-47816c6fc3ca\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54bb49d69d-864rn" Jul 15 16:57:36.457380 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.457327 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgc7j\" (UniqueName: \"kubernetes.io/projected/c3446ceb-be73-4e8a-8393-47816c6fc3ca-kube-api-access-hgc7j\") pod \"managed-serviceaccount-addon-agent-54bb49d69d-864rn\" (UID: \"c3446ceb-be73-4e8a-8393-47816c6fc3ca\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54bb49d69d-864rn" Jul 15 16:57:36.457443 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.457380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3446ceb-be73-4e8a-8393-47816c6fc3ca-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-54bb49d69d-864rn\" (UID: \"c3446ceb-be73-4e8a-8393-47816c6fc3ca\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54bb49d69d-864rn" Jul 15 16:57:36.459701 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.459673 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3446ceb-be73-4e8a-8393-47816c6fc3ca-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-54bb49d69d-864rn\" (UID: \"c3446ceb-be73-4e8a-8393-47816c6fc3ca\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54bb49d69d-864rn" Jul 15 16:57:36.465253 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.465236 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgc7j\" (UniqueName: \"kubernetes.io/projected/c3446ceb-be73-4e8a-8393-47816c6fc3ca-kube-api-access-hgc7j\") pod \"managed-serviceaccount-addon-agent-54bb49d69d-864rn\" (UID: \"c3446ceb-be73-4e8a-8393-47816c6fc3ca\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54bb49d69d-864rn" Jul 15 16:57:36.502299 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.502268 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54bb49d69d-864rn" Jul 15 16:57:36.613924 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.613890 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54bb49d69d-864rn"] Jul 15 16:57:36.618165 ip-10-0-134-118 kubenswrapper[2562]: W0715 16:57:36.618139 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3446ceb_be73_4e8a_8393_47816c6fc3ca.slice/crio-959882659bc7efce7ecf8a2c40a66774e54a1b659e8e5e46c4f8ed3843ce74de WatchSource:0}: Error finding container 959882659bc7efce7ecf8a2c40a66774e54a1b659e8e5e46c4f8ed3843ce74de: Status 404 returned error can't find the container with id 959882659bc7efce7ecf8a2c40a66774e54a1b659e8e5e46c4f8ed3843ce74de Jul 15 16:57:36.853767 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:36.853686 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54bb49d69d-864rn" event={"ID":"c3446ceb-be73-4e8a-8393-47816c6fc3ca","Type":"ContainerStarted","Data":"959882659bc7efce7ecf8a2c40a66774e54a1b659e8e5e46c4f8ed3843ce74de"} Jul 15 16:57:39.862091 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:39.862051 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54bb49d69d-864rn" event={"ID":"c3446ceb-be73-4e8a-8393-47816c6fc3ca","Type":"ContainerStarted","Data":"8cf9a58f3f340fd0ff8483500b52be91bae51572d30b5daa825c6f2df8a9d93a"} Jul 15 16:57:39.878227 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:57:39.878181 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54bb49d69d-864rn" podStartSLOduration=1.499366446 podStartE2EDuration="3.878167465s" totalImagesPullingTime="2.378801019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:57:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:39.877426353 +0000 UTC m=+75.926423547" watchObservedRunningTime="2026-07-15 16:57:39.878167465 +0000 UTC m=+75.927164655" Jul 15 16:58:02.223787 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:58:02.223676 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:58:02.224260 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:58:02.223808 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:58:02.224260 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:58:02.223874 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls podName:65c4ae32-87b7-46f2-8370-510449c1020e nodeName:}" failed. No retries permitted until 2026-07-15 16:59:06.223859916 +0000 UTC m=+162.272857086 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls") pod "dns-default-dph8q" (UID: "65c4ae32-87b7-46f2-8370-510449c1020e") : secret "dns-default-metrics-tls" not found Jul 15 16:58:05.851954 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:58:05.851928 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-btjr2" Jul 15 16:58:33.118887 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:58:33.118845 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs\") pod \"network-metrics-daemon-mbqkc\" (UID: \"4f43f08e-aa70-42d2-9207-93156d956d7f\") " pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:58:33.119389 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:58:33.118957 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 15 16:58:33.119389 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:58:33.119019 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs podName:4f43f08e-aa70-42d2-9207-93156d956d7f nodeName:}" failed. No retries permitted until 2026-07-15 17:00:35.119005629 +0000 UTC m=+251.168002801 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs") pod "network-metrics-daemon-mbqkc" (UID: "4f43f08e-aa70-42d2-9207-93156d956d7f") : secret "metrics-daemon-secret" not found Jul 15 16:59:01.403470 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:59:01.403421 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-dph8q" podUID="65c4ae32-87b7-46f2-8370-510449c1020e" Jul 15 16:59:01.580584 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:59:01.580537 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 16:59:02.047854 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:59:02.047826 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dph8q" Jul 15 16:59:06.235867 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:59:06.235829 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 16:59:06.236237 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:59:06.235950 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:59:06.236237 ip-10-0-134-118 kubenswrapper[2562]: E0715 16:59:06.235999 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls podName:65c4ae32-87b7-46f2-8370-510449c1020e nodeName:}" failed. No retries permitted until 2026-07-15 17:01:08.235987061 +0000 UTC m=+284.284984231 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls") pod "dns-default-dph8q" (UID: "65c4ae32-87b7-46f2-8370-510449c1020e") : secret "dns-default-metrics-tls" not found Jul 15 16:59:12.560946 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:59:12.560907 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 16:59:39.136155 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:59:39.136128 2562 generic.go:356] "Generic (PLEG): container finished" podID="c3446ceb-be73-4e8a-8393-47816c6fc3ca" containerID="8cf9a58f3f340fd0ff8483500b52be91bae51572d30b5daa825c6f2df8a9d93a" exitCode=255 Jul 15 16:59:39.136427 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:59:39.136172 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54bb49d69d-864rn" event={"ID":"c3446ceb-be73-4e8a-8393-47816c6fc3ca","Type":"ContainerDied","Data":"8cf9a58f3f340fd0ff8483500b52be91bae51572d30b5daa825c6f2df8a9d93a"} Jul 15 16:59:39.136468 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:59:39.136438 2562 scope.go:117] "RemoveContainer" containerID="8cf9a58f3f340fd0ff8483500b52be91bae51572d30b5daa825c6f2df8a9d93a" Jul 15 16:59:40.142726 ip-10-0-134-118 kubenswrapper[2562]: I0715 16:59:40.142692 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54bb49d69d-864rn" event={"ID":"c3446ceb-be73-4e8a-8393-47816c6fc3ca","Type":"ContainerStarted","Data":"0da02227bdff833816a8c09c64fd9ef0acd385da1b11b8e09030533a6c00c678"} Jul 15 17:00:35.188524 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:35.188489 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs\") pod \"network-metrics-daemon-mbqkc\" (UID: \"4f43f08e-aa70-42d2-9207-93156d956d7f\") " pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 17:00:35.188990 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:35.188606 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 15 17:00:35.188990 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:35.188656 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs podName:4f43f08e-aa70-42d2-9207-93156d956d7f nodeName:}" failed. No retries permitted until 2026-07-15 17:02:37.188642574 +0000 UTC m=+373.237639744 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs") pod "network-metrics-daemon-mbqkc" (UID: "4f43f08e-aa70-42d2-9207-93156d956d7f") : secret "metrics-daemon-secret" not found Jul 15 17:00:37.942175 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.942143 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz"] Jul 15 17:00:37.944023 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.944008 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz" Jul 15 17:00:37.946279 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.946220 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-djhfs"] Jul 15 17:00:37.946416 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.946315 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 15 17:00:37.946416 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.946321 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 15 17:00:37.946630 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.946613 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 15 17:00:37.946847 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.946828 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 15 17:00:37.947332 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.947313 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-6lpd2\"" Jul 15 17:00:37.948379 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.948363 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-68ddcc8458-j29rh"] Jul 15 17:00:37.948475 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.948465 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:00:37.950274 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.950256 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7984794bd6-p4vld"] Jul 15 17:00:37.950388 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.950362 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:37.951236 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.951216 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 15 17:00:37.951344 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.951253 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 15 17:00:37.951344 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.951279 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 15 17:00:37.951580 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.951567 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 15 17:00:37.952098 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.952078 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:37.952492 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.952474 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 17:00:37.953424 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.953406 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 15 17:00:37.953520 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.953443 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 15 17:00:37.953520 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.953406 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 15 17:00:37.953633 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.953517 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j5ccs\"" Jul 15 17:00:37.953680 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.953646 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 15 17:00:37.953680 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.953669 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-928j5\"" Jul 15 17:00:37.953783 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.953766 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 17:00:37.955690 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.955673 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 17:00:37.955764 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.955736 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 17:00:37.955999 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.955985 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 17:00:37.956409 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.956395 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hhz8f\"" Jul 15 17:00:37.963433 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.963394 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 15 17:00:37.964050 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.964027 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz"] Jul 15 17:00:37.966810 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.966785 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 17:00:37.966977 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.966962 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7984794bd6-p4vld"] Jul 15 17:00:37.967970 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.967924 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68ddcc8458-j29rh"] Jul 15 17:00:37.978318 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:37.978280 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-djhfs"] Jul 15 17:00:38.109710 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.109665 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lmt2\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-kube-api-access-4lmt2\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.109890 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.109716 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7bb56c5-b728-49f2-a7d8-e869c083e49f-trusted-ca\") pod \"console-operator-7866497494-djhfs\" (UID: \"a7bb56c5-b728-49f2-a7d8-e869c083e49f\") " pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:00:38.109890 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.109754 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-ca-trust-extracted\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.109890 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.109782 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-certificates\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.109890 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.109800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-bound-sa-token\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.109890 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.109818 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.109890 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.109833 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-stats-auth\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:38.109890 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.109867 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hrvw\" (UniqueName: \"kubernetes.io/projected/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-kube-api-access-7hrvw\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:38.109890 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.109889 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-default-certificate\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:38.110171 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.109921 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:38.110171 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.109937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-installation-pull-secrets\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.110171 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.109955 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c7ffc33d-460d-437f-baf0-c93e7efc4dff-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-27gvz\" (UID: \"c7ffc33d-460d-437f-baf0-c93e7efc4dff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz" Jul 15 17:00:38.110171 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.109977 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgv4r\" (UniqueName: \"kubernetes.io/projected/c7ffc33d-460d-437f-baf0-c93e7efc4dff-kube-api-access-xgv4r\") pod \"kube-storage-version-migrator-operator-799cb7f958-27gvz\" (UID: \"c7ffc33d-460d-437f-baf0-c93e7efc4dff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz" Jul 15 17:00:38.110171 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.109996 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-trusted-ca\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.110171 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.110025 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7bb56c5-b728-49f2-a7d8-e869c083e49f-serving-cert\") pod \"console-operator-7866497494-djhfs\" (UID: \"a7bb56c5-b728-49f2-a7d8-e869c083e49f\") " pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:00:38.110171 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.110069 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-image-registry-private-configuration\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.110171 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.110087 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:38.110171 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.110108 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7bb56c5-b728-49f2-a7d8-e869c083e49f-config\") pod \"console-operator-7866497494-djhfs\" (UID: \"a7bb56c5-b728-49f2-a7d8-e869c083e49f\") " pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:00:38.110171 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.110124 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb84h\" (UniqueName: \"kubernetes.io/projected/a7bb56c5-b728-49f2-a7d8-e869c083e49f-kube-api-access-fb84h\") pod \"console-operator-7866497494-djhfs\" (UID: \"a7bb56c5-b728-49f2-a7d8-e869c083e49f\") " pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:00:38.110501 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.110186 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c7ffc33d-460d-437f-baf0-c93e7efc4dff-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-27gvz\" (UID: \"c7ffc33d-460d-437f-baf0-c93e7efc4dff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz" Jul 15 17:00:38.211064 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.210979 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-installation-pull-secrets\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.211064 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211024 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c7ffc33d-460d-437f-baf0-c93e7efc4dff-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-27gvz\" (UID: \"c7ffc33d-460d-437f-baf0-c93e7efc4dff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz" Jul 15 17:00:38.211064 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211052 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgv4r\" (UniqueName: \"kubernetes.io/projected/c7ffc33d-460d-437f-baf0-c93e7efc4dff-kube-api-access-xgv4r\") pod \"kube-storage-version-migrator-operator-799cb7f958-27gvz\" (UID: \"c7ffc33d-460d-437f-baf0-c93e7efc4dff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz" Jul 15 17:00:38.211332 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211083 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-trusted-ca\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.211332 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211103 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7bb56c5-b728-49f2-a7d8-e869c083e49f-serving-cert\") pod \"console-operator-7866497494-djhfs\" (UID: \"a7bb56c5-b728-49f2-a7d8-e869c083e49f\") " pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:00:38.211332 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211127 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-image-registry-private-configuration\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.211332 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211153 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:38.211332 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211184 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7bb56c5-b728-49f2-a7d8-e869c083e49f-config\") pod \"console-operator-7866497494-djhfs\" (UID: \"a7bb56c5-b728-49f2-a7d8-e869c083e49f\") " pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:00:38.211332 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb84h\" (UniqueName: \"kubernetes.io/projected/a7bb56c5-b728-49f2-a7d8-e869c083e49f-kube-api-access-fb84h\") pod \"console-operator-7866497494-djhfs\" (UID: \"a7bb56c5-b728-49f2-a7d8-e869c083e49f\") " pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:00:38.211332 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211243 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c7ffc33d-460d-437f-baf0-c93e7efc4dff-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-27gvz\" (UID: \"c7ffc33d-460d-437f-baf0-c93e7efc4dff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz" Jul 15 17:00:38.211332 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211272 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lmt2\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-kube-api-access-4lmt2\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.211332 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211320 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7bb56c5-b728-49f2-a7d8-e869c083e49f-trusted-ca\") pod \"console-operator-7866497494-djhfs\" (UID: \"a7bb56c5-b728-49f2-a7d8-e869c083e49f\") " pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:00:38.211730 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211351 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-ca-trust-extracted\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.211730 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211382 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-certificates\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.211730 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211404 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-bound-sa-token\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.211730 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211428 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.211730 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211450 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-stats-auth\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:38.211730 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211494 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hrvw\" (UniqueName: \"kubernetes.io/projected/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-kube-api-access-7hrvw\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:38.211730 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211521 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-default-certificate\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:38.211730 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211559 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:38.211730 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.211671 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c7ffc33d-460d-437f-baf0-c93e7efc4dff-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-27gvz\" (UID: \"c7ffc33d-460d-437f-baf0-c93e7efc4dff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz" Jul 15 17:00:38.211730 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:38.211705 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle podName:afcb1c98-744f-4fbe-8396-3217c0d7f2ba nodeName:}" failed. No retries permitted until 2026-07-15 17:00:38.711686116 +0000 UTC m=+254.760683287 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle") pod "router-default-68ddcc8458-j29rh" (UID: "afcb1c98-744f-4fbe-8396-3217c0d7f2ba") : configmap references non-existent config key: service-ca.crt Jul 15 17:00:38.211730 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:38.211711 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 17:00:38.212397 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:38.211775 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs podName:afcb1c98-744f-4fbe-8396-3217c0d7f2ba nodeName:}" failed. No retries permitted until 2026-07-15 17:00:38.711757135 +0000 UTC m=+254.760754306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs") pod "router-default-68ddcc8458-j29rh" (UID: "afcb1c98-744f-4fbe-8396-3217c0d7f2ba") : secret "router-metrics-certs-default" not found Jul 15 17:00:38.212397 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.212201 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-trusted-ca\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.212513 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.212482 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7bb56c5-b728-49f2-a7d8-e869c083e49f-config\") pod \"console-operator-7866497494-djhfs\" (UID: \"a7bb56c5-b728-49f2-a7d8-e869c083e49f\") " pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:00:38.213132 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:38.212704 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 17:00:38.213132 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:38.212723 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7984794bd6-p4vld: secret "image-registry-tls" not found Jul 15 17:00:38.213132 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:38.212776 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls podName:5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c nodeName:}" failed. No retries permitted until 2026-07-15 17:00:38.712761563 +0000 UTC m=+254.761758746 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls") pod "image-registry-7984794bd6-p4vld" (UID: "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c") : secret "image-registry-tls" not found Jul 15 17:00:38.213132 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.212705 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7bb56c5-b728-49f2-a7d8-e869c083e49f-trusted-ca\") pod \"console-operator-7866497494-djhfs\" (UID: \"a7bb56c5-b728-49f2-a7d8-e869c083e49f\") " pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:00:38.213132 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.213089 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-certificates\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.213759 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.213715 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-ca-trust-extracted\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.214022 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.213999 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7bb56c5-b728-49f2-a7d8-e869c083e49f-serving-cert\") pod \"console-operator-7866497494-djhfs\" (UID: \"a7bb56c5-b728-49f2-a7d8-e869c083e49f\") " pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:00:38.214195 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.214163 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-installation-pull-secrets\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.214364 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.214340 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-image-registry-private-configuration\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.214535 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.214519 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-default-certificate\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:38.214688 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.214673 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c7ffc33d-460d-437f-baf0-c93e7efc4dff-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-27gvz\" (UID: \"c7ffc33d-460d-437f-baf0-c93e7efc4dff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz" Jul 15 17:00:38.214950 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.214935 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-stats-auth\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:38.219275 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.219248 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hrvw\" (UniqueName: \"kubernetes.io/projected/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-kube-api-access-7hrvw\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:38.219389 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.219354 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lmt2\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-kube-api-access-4lmt2\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.220140 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.220115 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-bound-sa-token\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.220211 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.220182 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgv4r\" (UniqueName: \"kubernetes.io/projected/c7ffc33d-460d-437f-baf0-c93e7efc4dff-kube-api-access-xgv4r\") pod \"kube-storage-version-migrator-operator-799cb7f958-27gvz\" (UID: \"c7ffc33d-460d-437f-baf0-c93e7efc4dff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz" Jul 15 17:00:38.220471 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.220453 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb84h\" (UniqueName: \"kubernetes.io/projected/a7bb56c5-b728-49f2-a7d8-e869c083e49f-kube-api-access-fb84h\") pod \"console-operator-7866497494-djhfs\" (UID: \"a7bb56c5-b728-49f2-a7d8-e869c083e49f\") " pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:00:38.254821 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.254793 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz" Jul 15 17:00:38.262724 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.262680 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:00:38.378517 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.378486 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz"] Jul 15 17:00:38.381259 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:00:38.381211 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7ffc33d_460d_437f_baf0_c93e7efc4dff.slice/crio-776a903e5ed969f83829f6e7e0fbd25a1f30d5b0badce80fb70f0dee1d97e732 WatchSource:0}: Error finding container 776a903e5ed969f83829f6e7e0fbd25a1f30d5b0badce80fb70f0dee1d97e732: Status 404 returned error can't find the container with id 776a903e5ed969f83829f6e7e0fbd25a1f30d5b0badce80fb70f0dee1d97e732 Jul 15 17:00:38.394145 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.394120 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-djhfs"] Jul 15 17:00:38.397424 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:00:38.397393 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7bb56c5_b728_49f2_a7d8_e869c083e49f.slice/crio-27f8f2d2fcde90a67f199e6af571e4ba3dfb40388f66fd014bf2b224450d34d9 WatchSource:0}: Error finding container 27f8f2d2fcde90a67f199e6af571e4ba3dfb40388f66fd014bf2b224450d34d9: Status 404 returned error can't find the container with id 27f8f2d2fcde90a67f199e6af571e4ba3dfb40388f66fd014bf2b224450d34d9 Jul 15 17:00:38.716392 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.716359 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:38.716552 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.716416 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:38.716552 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:38.716513 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 17:00:38.716626 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:38.716559 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:38.716626 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:38.716579 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs podName:afcb1c98-744f-4fbe-8396-3217c0d7f2ba nodeName:}" failed. No retries permitted until 2026-07-15 17:00:39.716558977 +0000 UTC m=+255.765556166 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs") pod "router-default-68ddcc8458-j29rh" (UID: "afcb1c98-744f-4fbe-8396-3217c0d7f2ba") : secret "router-metrics-certs-default" not found Jul 15 17:00:38.716626 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:38.716515 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 17:00:38.716626 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:38.716603 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7984794bd6-p4vld: secret "image-registry-tls" not found Jul 15 17:00:38.716761 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:38.716656 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle podName:afcb1c98-744f-4fbe-8396-3217c0d7f2ba nodeName:}" failed. No retries permitted until 2026-07-15 17:00:39.716643592 +0000 UTC m=+255.765640774 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle") pod "router-default-68ddcc8458-j29rh" (UID: "afcb1c98-744f-4fbe-8396-3217c0d7f2ba") : configmap references non-existent config key: service-ca.crt Jul 15 17:00:38.716761 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:38.716677 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls podName:5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c nodeName:}" failed. No retries permitted until 2026-07-15 17:00:39.716668198 +0000 UTC m=+255.765665367 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls") pod "image-registry-7984794bd6-p4vld" (UID: "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c") : secret "image-registry-tls" not found Jul 15 17:00:39.306645 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:39.306594 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz" event={"ID":"c7ffc33d-460d-437f-baf0-c93e7efc4dff","Type":"ContainerStarted","Data":"776a903e5ed969f83829f6e7e0fbd25a1f30d5b0badce80fb70f0dee1d97e732"} Jul 15 17:00:39.307979 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:39.307951 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-djhfs" event={"ID":"a7bb56c5-b728-49f2-a7d8-e869c083e49f","Type":"ContainerStarted","Data":"27f8f2d2fcde90a67f199e6af571e4ba3dfb40388f66fd014bf2b224450d34d9"} Jul 15 17:00:39.723544 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:39.723502 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:39.723748 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:39.723582 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:39.723748 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:39.723620 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:39.723748 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:39.723643 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 17:00:39.723748 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:39.723669 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7984794bd6-p4vld: secret "image-registry-tls" not found Jul 15 17:00:39.723748 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:39.723732 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls podName:5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c nodeName:}" failed. No retries permitted until 2026-07-15 17:00:41.723712182 +0000 UTC m=+257.772709364 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls") pod "image-registry-7984794bd6-p4vld" (UID: "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c") : secret "image-registry-tls" not found Jul 15 17:00:39.724062 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:39.723810 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle podName:afcb1c98-744f-4fbe-8396-3217c0d7f2ba nodeName:}" failed. No retries permitted until 2026-07-15 17:00:41.723793248 +0000 UTC m=+257.772790443 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle") pod "router-default-68ddcc8458-j29rh" (UID: "afcb1c98-744f-4fbe-8396-3217c0d7f2ba") : configmap references non-existent config key: service-ca.crt Jul 15 17:00:39.724062 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:39.723851 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 17:00:39.724062 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:39.723908 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs podName:afcb1c98-744f-4fbe-8396-3217c0d7f2ba nodeName:}" failed. No retries permitted until 2026-07-15 17:00:41.723893166 +0000 UTC m=+257.772890341 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs") pod "router-default-68ddcc8458-j29rh" (UID: "afcb1c98-744f-4fbe-8396-3217c0d7f2ba") : secret "router-metrics-certs-default" not found Jul 15 17:00:41.315843 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:41.315752 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz" event={"ID":"c7ffc33d-460d-437f-baf0-c93e7efc4dff","Type":"ContainerStarted","Data":"9d37ea099ed08830f375887b26466967761a6c407ab0b2d982e0319bddf15a7b"} Jul 15 17:00:41.317359 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:41.317339 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/0.log" Jul 15 17:00:41.317450 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:41.317377 2562 generic.go:356] "Generic (PLEG): container finished" podID="a7bb56c5-b728-49f2-a7d8-e869c083e49f" containerID="f9cbfe598e350e7b7a2d2c65f004ed7eea4fe0bd4164888a62cfacd5ab756d4f" exitCode=255 Jul 15 17:00:41.317500 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:41.317445 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-djhfs" event={"ID":"a7bb56c5-b728-49f2-a7d8-e869c083e49f","Type":"ContainerDied","Data":"f9cbfe598e350e7b7a2d2c65f004ed7eea4fe0bd4164888a62cfacd5ab756d4f"} Jul 15 17:00:41.317642 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:41.317630 2562 scope.go:117] "RemoveContainer" containerID="f9cbfe598e350e7b7a2d2c65f004ed7eea4fe0bd4164888a62cfacd5ab756d4f" Jul 15 17:00:41.332473 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:41.332428 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz" podStartSLOduration=1.675027172 podStartE2EDuration="4.332411532s" totalImagesPullingTime="2.65738436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:41.331884135 +0000 UTC m=+257.380881327" watchObservedRunningTime="2026-07-15 17:00:41.332411532 +0000 UTC m=+257.381408723" Jul 15 17:00:41.740961 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:41.740932 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:41.741125 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:41.740995 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:41.741125 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:41.741031 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:41.741125 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:41.741083 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 17:00:41.741125 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:41.741105 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7984794bd6-p4vld: secret "image-registry-tls" not found Jul 15 17:00:41.741327 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:41.741158 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 17:00:41.741327 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:41.741173 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle podName:afcb1c98-744f-4fbe-8396-3217c0d7f2ba nodeName:}" failed. No retries permitted until 2026-07-15 17:00:45.741153905 +0000 UTC m=+261.790151080 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle") pod "router-default-68ddcc8458-j29rh" (UID: "afcb1c98-744f-4fbe-8396-3217c0d7f2ba") : configmap references non-existent config key: service-ca.crt Jul 15 17:00:41.741327 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:41.741191 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls podName:5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c nodeName:}" failed. No retries permitted until 2026-07-15 17:00:45.741183857 +0000 UTC m=+261.790181026 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls") pod "image-registry-7984794bd6-p4vld" (UID: "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c") : secret "image-registry-tls" not found Jul 15 17:00:41.741327 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:41.741221 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs podName:afcb1c98-744f-4fbe-8396-3217c0d7f2ba nodeName:}" failed. No retries permitted until 2026-07-15 17:00:45.741204312 +0000 UTC m=+261.790201488 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs") pod "router-default-68ddcc8458-j29rh" (UID: "afcb1c98-744f-4fbe-8396-3217c0d7f2ba") : secret "router-metrics-certs-default" not found Jul 15 17:00:42.322414 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:42.322386 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:00:42.322825 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:42.322808 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/0.log" Jul 15 17:00:42.322884 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:42.322842 2562 generic.go:356] "Generic (PLEG): container finished" podID="a7bb56c5-b728-49f2-a7d8-e869c083e49f" containerID="1cdbe9bf15e5c713f4e130ef4e5cdfadd273e555880ed991987aa7a9980df1c7" exitCode=255 Jul 15 17:00:42.322961 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:42.322940 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-djhfs" event={"ID":"a7bb56c5-b728-49f2-a7d8-e869c083e49f","Type":"ContainerDied","Data":"1cdbe9bf15e5c713f4e130ef4e5cdfadd273e555880ed991987aa7a9980df1c7"} Jul 15 17:00:42.323018 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:42.323012 2562 scope.go:117] "RemoveContainer" containerID="f9cbfe598e350e7b7a2d2c65f004ed7eea4fe0bd4164888a62cfacd5ab756d4f" Jul 15 17:00:42.323187 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:42.323172 2562 scope.go:117] "RemoveContainer" containerID="1cdbe9bf15e5c713f4e130ef4e5cdfadd273e555880ed991987aa7a9980df1c7" Jul 15 17:00:42.323396 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:42.323372 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-djhfs_openshift-console-operator(a7bb56c5-b728-49f2-a7d8-e869c083e49f)\"" pod="openshift-console-operator/console-operator-7866497494-djhfs" podUID="a7bb56c5-b728-49f2-a7d8-e869c083e49f" Jul 15 17:00:43.014590 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.014559 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vz9xc"] Jul 15 17:00:43.017320 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.017285 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:43.020074 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.020052 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 17:00:43.020883 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.020867 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 17:00:43.020976 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.020906 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 17:00:43.021202 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.021186 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 17:00:43.021961 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.021945 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tmfrj\"" Jul 15 17:00:43.036365 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.036346 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vz9xc"] Jul 15 17:00:43.153940 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.153909 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5c38aa01-993a-4d62-aeaa-87020199fca1-crio-socket\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:43.153940 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.153939 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c38aa01-993a-4d62-aeaa-87020199fca1-data-volume\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:43.154091 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.153959 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:43.154091 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.154031 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggbdt\" (UniqueName: \"kubernetes.io/projected/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-api-access-ggbdt\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:43.154091 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.154056 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:43.255398 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.255366 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5c38aa01-993a-4d62-aeaa-87020199fca1-crio-socket\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:43.255398 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.255401 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c38aa01-993a-4d62-aeaa-87020199fca1-data-volume\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:43.255517 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.255421 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:43.255517 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.255490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggbdt\" (UniqueName: \"kubernetes.io/projected/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-api-access-ggbdt\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:43.255517 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.255500 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5c38aa01-993a-4d62-aeaa-87020199fca1-crio-socket\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:43.255641 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.255526 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:43.255641 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:43.255586 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 17:00:43.255641 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:43.255641 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm podName:5c38aa01-993a-4d62-aeaa-87020199fca1 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:43.755628107 +0000 UTC m=+259.804625277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vz9xc" (UID: "5c38aa01-993a-4d62-aeaa-87020199fca1") : configmap "kube-rbac-proxy" not found Jul 15 17:00:43.255791 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:43.255709 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 17:00:43.255791 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.255733 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c38aa01-993a-4d62-aeaa-87020199fca1-data-volume\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:43.255791 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:43.255777 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls podName:5c38aa01-993a-4d62-aeaa-87020199fca1 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:43.755761047 +0000 UTC m=+259.804758217 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vz9xc" (UID: "5c38aa01-993a-4d62-aeaa-87020199fca1") : secret "insights-runtime-extractor-tls" not found Jul 15 17:00:43.264160 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.264136 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggbdt\" (UniqueName: \"kubernetes.io/projected/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-api-access-ggbdt\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:43.327457 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.327394 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:00:43.327751 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.327695 2562 scope.go:117] "RemoveContainer" containerID="1cdbe9bf15e5c713f4e130ef4e5cdfadd273e555880ed991987aa7a9980df1c7" Jul 15 17:00:43.328232 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:43.328192 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-djhfs_openshift-console-operator(a7bb56c5-b728-49f2-a7d8-e869c083e49f)\"" pod="openshift-console-operator/console-operator-7866497494-djhfs" podUID="a7bb56c5-b728-49f2-a7d8-e869c083e49f" Jul 15 17:00:43.759497 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.759468 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:43.759642 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:43.759540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:43.759642 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:43.759601 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 17:00:43.759704 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:43.759649 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm podName:5c38aa01-993a-4d62-aeaa-87020199fca1 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:44.759636417 +0000 UTC m=+260.808633586 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vz9xc" (UID: "5c38aa01-993a-4d62-aeaa-87020199fca1") : configmap "kube-rbac-proxy" not found Jul 15 17:00:43.759704 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:43.759601 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 17:00:43.759769 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:43.759729 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls podName:5c38aa01-993a-4d62-aeaa-87020199fca1 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:44.759715997 +0000 UTC m=+260.808713166 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vz9xc" (UID: "5c38aa01-993a-4d62-aeaa-87020199fca1") : secret "insights-runtime-extractor-tls" not found Jul 15 17:00:44.648912 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.648884 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-ws2km"] Jul 15 17:00:44.650890 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.650873 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-ws2km" Jul 15 17:00:44.653110 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.653091 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 15 17:00:44.653472 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.653454 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 15 17:00:44.653580 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.653518 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 15 17:00:44.653995 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.653980 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 15 17:00:44.654070 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.654015 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-fgdhp\"" Jul 15 17:00:44.660052 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.660023 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-ws2km"] Jul 15 17:00:44.664663 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.664643 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6dded9c2-8884-45b0-bf8f-e29d6ff31ea1-signing-cabundle\") pod \"service-ca-79f68b9f4f-ws2km\" (UID: \"6dded9c2-8884-45b0-bf8f-e29d6ff31ea1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ws2km" Jul 15 17:00:44.664758 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.664672 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kn2v\" (UniqueName: \"kubernetes.io/projected/6dded9c2-8884-45b0-bf8f-e29d6ff31ea1-kube-api-access-9kn2v\") pod \"service-ca-79f68b9f4f-ws2km\" (UID: \"6dded9c2-8884-45b0-bf8f-e29d6ff31ea1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ws2km" Jul 15 17:00:44.664818 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.664776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6dded9c2-8884-45b0-bf8f-e29d6ff31ea1-signing-key\") pod \"service-ca-79f68b9f4f-ws2km\" (UID: \"6dded9c2-8884-45b0-bf8f-e29d6ff31ea1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ws2km" Jul 15 17:00:44.765485 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.765460 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:44.765580 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.765536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6dded9c2-8884-45b0-bf8f-e29d6ff31ea1-signing-cabundle\") pod \"service-ca-79f68b9f4f-ws2km\" (UID: \"6dded9c2-8884-45b0-bf8f-e29d6ff31ea1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ws2km" Jul 15 17:00:44.765580 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.765563 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kn2v\" (UniqueName: \"kubernetes.io/projected/6dded9c2-8884-45b0-bf8f-e29d6ff31ea1-kube-api-access-9kn2v\") pod \"service-ca-79f68b9f4f-ws2km\" (UID: \"6dded9c2-8884-45b0-bf8f-e29d6ff31ea1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ws2km" Jul 15 17:00:44.765654 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.765588 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:44.765654 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:44.765601 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 17:00:44.765728 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.765661 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6dded9c2-8884-45b0-bf8f-e29d6ff31ea1-signing-key\") pod \"service-ca-79f68b9f4f-ws2km\" (UID: \"6dded9c2-8884-45b0-bf8f-e29d6ff31ea1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ws2km" Jul 15 17:00:44.765728 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:44.765676 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls podName:5c38aa01-993a-4d62-aeaa-87020199fca1 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:46.765658754 +0000 UTC m=+262.814655927 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vz9xc" (UID: "5c38aa01-993a-4d62-aeaa-87020199fca1") : secret "insights-runtime-extractor-tls" not found Jul 15 17:00:44.765802 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:44.765741 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 17:00:44.765844 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:44.765811 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm podName:5c38aa01-993a-4d62-aeaa-87020199fca1 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:46.765794687 +0000 UTC m=+262.814791857 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vz9xc" (UID: "5c38aa01-993a-4d62-aeaa-87020199fca1") : configmap "kube-rbac-proxy" not found Jul 15 17:00:44.766241 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.766223 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6dded9c2-8884-45b0-bf8f-e29d6ff31ea1-signing-cabundle\") pod \"service-ca-79f68b9f4f-ws2km\" (UID: \"6dded9c2-8884-45b0-bf8f-e29d6ff31ea1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ws2km" Jul 15 17:00:44.767932 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.767913 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6dded9c2-8884-45b0-bf8f-e29d6ff31ea1-signing-key\") pod \"service-ca-79f68b9f4f-ws2km\" (UID: \"6dded9c2-8884-45b0-bf8f-e29d6ff31ea1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ws2km" Jul 15 17:00:44.773858 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.773836 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kn2v\" (UniqueName: \"kubernetes.io/projected/6dded9c2-8884-45b0-bf8f-e29d6ff31ea1-kube-api-access-9kn2v\") pod \"service-ca-79f68b9f4f-ws2km\" (UID: \"6dded9c2-8884-45b0-bf8f-e29d6ff31ea1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ws2km" Jul 15 17:00:44.960034 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:44.960012 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-ws2km" Jul 15 17:00:45.078285 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:45.078246 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-ws2km"] Jul 15 17:00:45.081786 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:00:45.081754 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dded9c2_8884_45b0_bf8f_e29d6ff31ea1.slice/crio-abe77d5bda85e6a5f2fa9c06f348a64fb9cfc35be3fdaeaff1de65b8ef1daa62 WatchSource:0}: Error finding container abe77d5bda85e6a5f2fa9c06f348a64fb9cfc35be3fdaeaff1de65b8ef1daa62: Status 404 returned error can't find the container with id abe77d5bda85e6a5f2fa9c06f348a64fb9cfc35be3fdaeaff1de65b8ef1daa62 Jul 15 17:00:45.334072 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:45.333992 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-ws2km" event={"ID":"6dded9c2-8884-45b0-bf8f-e29d6ff31ea1","Type":"ContainerStarted","Data":"abe77d5bda85e6a5f2fa9c06f348a64fb9cfc35be3fdaeaff1de65b8ef1daa62"} Jul 15 17:00:45.771569 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:45.771543 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:45.771967 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:45.771591 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:45.771967 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:45.771624 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:45.771967 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:45.771706 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 17:00:45.771967 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:45.771708 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 17:00:45.771967 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:45.771728 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7984794bd6-p4vld: secret "image-registry-tls" not found Jul 15 17:00:45.771967 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:45.771762 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs podName:afcb1c98-744f-4fbe-8396-3217c0d7f2ba nodeName:}" failed. No retries permitted until 2026-07-15 17:00:53.771745548 +0000 UTC m=+269.820742717 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs") pod "router-default-68ddcc8458-j29rh" (UID: "afcb1c98-744f-4fbe-8396-3217c0d7f2ba") : secret "router-metrics-certs-default" not found Jul 15 17:00:45.771967 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:45.771800 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls podName:5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c nodeName:}" failed. No retries permitted until 2026-07-15 17:00:53.771769876 +0000 UTC m=+269.820767050 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls") pod "image-registry-7984794bd6-p4vld" (UID: "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c") : secret "image-registry-tls" not found Jul 15 17:00:45.771967 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:45.771820 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle podName:afcb1c98-744f-4fbe-8396-3217c0d7f2ba nodeName:}" failed. No retries permitted until 2026-07-15 17:00:53.7718109 +0000 UTC m=+269.820808072 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle") pod "router-default-68ddcc8458-j29rh" (UID: "afcb1c98-744f-4fbe-8396-3217c0d7f2ba") : configmap references non-existent config key: service-ca.crt Jul 15 17:00:45.972240 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:45.972218 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tzkgv_cc6a2e63-80e9-44d9-9708-dc85bd571957/dns-node-resolver/0.log" Jul 15 17:00:46.773609 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:46.773530 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8w7fh_1fd95390-8b40-475c-ab93-ab641b315e09/node-ca/0.log" Jul 15 17:00:46.777825 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:46.777802 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:46.777926 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:46.777877 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:46.777978 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:46.777961 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 17:00:46.778024 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:46.778014 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls podName:5c38aa01-993a-4d62-aeaa-87020199fca1 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:50.778000705 +0000 UTC m=+266.826997876 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vz9xc" (UID: "5c38aa01-993a-4d62-aeaa-87020199fca1") : secret "insights-runtime-extractor-tls" not found Jul 15 17:00:46.778072 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:46.777964 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 17:00:46.778106 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:46.778073 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm podName:5c38aa01-993a-4d62-aeaa-87020199fca1 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:50.778062189 +0000 UTC m=+266.827059371 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vz9xc" (UID: "5c38aa01-993a-4d62-aeaa-87020199fca1") : configmap "kube-rbac-proxy" not found Jul 15 17:00:47.373244 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:47.373211 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jd6qq_f5d75ab3-c94f-4349-953a-05abad93f858/serve-healthcheck-canary/0.log" Jul 15 17:00:48.263225 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:48.263144 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:00:48.263225 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:48.263220 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:00:48.263618 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:48.263608 2562 scope.go:117] "RemoveContainer" containerID="1cdbe9bf15e5c713f4e130ef4e5cdfadd273e555880ed991987aa7a9980df1c7" Jul 15 17:00:48.263790 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:48.263769 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-djhfs_openshift-console-operator(a7bb56c5-b728-49f2-a7d8-e869c083e49f)\"" pod="openshift-console-operator/console-operator-7866497494-djhfs" podUID="a7bb56c5-b728-49f2-a7d8-e869c083e49f" Jul 15 17:00:48.344782 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:48.344751 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-ws2km" event={"ID":"6dded9c2-8884-45b0-bf8f-e29d6ff31ea1","Type":"ContainerStarted","Data":"d8c98bb4d483b773c391e9e3c6af5502676764131f0d35e0a5e1f111e049a5a9"} Jul 15 17:00:48.360817 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:48.360776 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-ws2km" podStartSLOduration=1.733839089 podStartE2EDuration="4.360763802s" totalImagesPullingTime="2.626924713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:48.360228667 +0000 UTC m=+264.409225870" watchObservedRunningTime="2026-07-15 17:00:48.360763802 +0000 UTC m=+264.409760994" Jul 15 17:00:48.375462 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:48.375439 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-27gvz_c7ffc33d-460d-437f-baf0-c93e7efc4dff/kube-storage-version-migrator-operator/0.log" Jul 15 17:00:50.808531 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:50.808498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:50.808875 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:50.808560 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:50.808875 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:50.808615 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 17:00:50.808875 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:50.808649 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 17:00:50.808875 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:50.808671 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm podName:5c38aa01-993a-4d62-aeaa-87020199fca1 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:58.808657471 +0000 UTC m=+274.857654641 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vz9xc" (UID: "5c38aa01-993a-4d62-aeaa-87020199fca1") : configmap "kube-rbac-proxy" not found Jul 15 17:00:50.808875 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:50.808688 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls podName:5c38aa01-993a-4d62-aeaa-87020199fca1 nodeName:}" failed. No retries permitted until 2026-07-15 17:00:58.808678319 +0000 UTC m=+274.857675489 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vz9xc" (UID: "5c38aa01-993a-4d62-aeaa-87020199fca1") : secret "insights-runtime-extractor-tls" not found Jul 15 17:00:53.833163 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:53.833132 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:53.833553 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:53.833183 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:53.833553 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:53.833331 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:00:53.833553 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:53.833338 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle podName:afcb1c98-744f-4fbe-8396-3217c0d7f2ba nodeName:}" failed. No retries permitted until 2026-07-15 17:01:09.833321353 +0000 UTC m=+285.882318523 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle") pod "router-default-68ddcc8458-j29rh" (UID: "afcb1c98-744f-4fbe-8396-3217c0d7f2ba") : configmap references non-existent config key: service-ca.crt Jul 15 17:00:53.833553 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:53.833459 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 17:00:53.833553 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:53.833512 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs podName:afcb1c98-744f-4fbe-8396-3217c0d7f2ba nodeName:}" failed. No retries permitted until 2026-07-15 17:01:09.833498559 +0000 UTC m=+285.882495742 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs") pod "router-default-68ddcc8458-j29rh" (UID: "afcb1c98-744f-4fbe-8396-3217c0d7f2ba") : secret "router-metrics-certs-default" not found Jul 15 17:00:53.835491 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:53.835464 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls\") pod \"image-registry-7984794bd6-p4vld\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:53.876351 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:53.876312 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:54.003218 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:54.003187 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7984794bd6-p4vld"] Jul 15 17:00:54.006507 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:00:54.006465 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ed58ef1_012e_47e7_b8c4_9625f4dd6e1c.slice/crio-ac4f93c5ac7c5b3cf87a1153c0ebe7a1c03b8240bafd9d7d2b942012bf721a76 WatchSource:0}: Error finding container ac4f93c5ac7c5b3cf87a1153c0ebe7a1c03b8240bafd9d7d2b942012bf721a76: Status 404 returned error can't find the container with id ac4f93c5ac7c5b3cf87a1153c0ebe7a1c03b8240bafd9d7d2b942012bf721a76 Jul 15 17:00:54.363549 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:54.363518 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7984794bd6-p4vld" event={"ID":"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c","Type":"ContainerStarted","Data":"ed7d4af041c3298128126c6d9e03be39a94ddd35d167020e0b0da198e96fcfe4"} Jul 15 17:00:54.363717 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:54.363573 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7984794bd6-p4vld" event={"ID":"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c","Type":"ContainerStarted","Data":"ac4f93c5ac7c5b3cf87a1153c0ebe7a1c03b8240bafd9d7d2b942012bf721a76"} Jul 15 17:00:54.363717 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:54.363634 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:00:54.385129 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:54.385086 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7984794bd6-p4vld" podStartSLOduration=17.38507161 podStartE2EDuration="17.38507161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:54.383450527 +0000 UTC m=+270.432447718" watchObservedRunningTime="2026-07-15 17:00:54.38507161 +0000 UTC m=+270.434068800" Jul 15 17:00:58.874233 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:58.874198 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:58.874592 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:58.874303 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:00:58.874592 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:58.874368 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 17:00:58.874592 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:00:58.874432 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls podName:5c38aa01-993a-4d62-aeaa-87020199fca1 nodeName:}" failed. No retries permitted until 2026-07-15 17:01:14.874417116 +0000 UTC m=+290.923414301 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vz9xc" (UID: "5c38aa01-993a-4d62-aeaa-87020199fca1") : secret "insights-runtime-extractor-tls" not found Jul 15 17:00:58.874882 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:00:58.874862 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c38aa01-993a-4d62-aeaa-87020199fca1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:01:01.561064 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:01.561031 2562 scope.go:117] "RemoveContainer" containerID="1cdbe9bf15e5c713f4e130ef4e5cdfadd273e555880ed991987aa7a9980df1c7" Jul 15 17:01:02.389800 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:02.389759 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:01:02.389984 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:02.389838 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-djhfs" event={"ID":"a7bb56c5-b728-49f2-a7d8-e869c083e49f","Type":"ContainerStarted","Data":"8ced7a5778ec110aad2011ad738741312c0ea0333213987c232c551d2ccf2ef4"} Jul 15 17:01:02.390422 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:02.390381 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:01:02.396255 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:02.396213 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-djhfs" Jul 15 17:01:02.410422 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:02.410380 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-djhfs" podStartSLOduration=22.7709711 podStartE2EDuration="25.410367508s" totalImagesPullingTime="2.639396408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:02.409332568 +0000 UTC m=+278.458329760" watchObservedRunningTime="2026-07-15 17:01:02.410367508 +0000 UTC m=+278.459364738" Jul 15 17:01:05.048907 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:01:05.048870 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-dph8q" podUID="65c4ae32-87b7-46f2-8370-510449c1020e" Jul 15 17:01:05.398982 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:05.398908 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dph8q" Jul 15 17:01:07.856199 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:07.856105 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7984794bd6-p4vld"] Jul 15 17:01:07.900388 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:07.900351 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67748f5cf7-2m5lh"] Jul 15 17:01:07.903041 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:07.903025 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:07.920173 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:07.920147 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67748f5cf7-2m5lh"] Jul 15 17:01:07.945026 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:07.944999 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2049531-ac65-45a6-89d4-fe638debd103-image-registry-private-configuration\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:07.945130 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:07.945032 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2049531-ac65-45a6-89d4-fe638debd103-registry-certificates\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:07.945130 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:07.945066 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjk4f\" (UniqueName: \"kubernetes.io/projected/f2049531-ac65-45a6-89d4-fe638debd103-kube-api-access-rjk4f\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:07.945202 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:07.945121 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2049531-ac65-45a6-89d4-fe638debd103-installation-pull-secrets\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:07.945202 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:07.945176 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2049531-ac65-45a6-89d4-fe638debd103-ca-trust-extracted\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:07.945270 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:07.945217 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2049531-ac65-45a6-89d4-fe638debd103-trusted-ca\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:07.945270 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:07.945265 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2049531-ac65-45a6-89d4-fe638debd103-registry-tls\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:07.945362 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:07.945311 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2049531-ac65-45a6-89d4-fe638debd103-bound-sa-token\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.046373 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.046342 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2049531-ac65-45a6-89d4-fe638debd103-trusted-ca\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.046526 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.046388 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2049531-ac65-45a6-89d4-fe638debd103-registry-tls\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.046576 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.046515 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2049531-ac65-45a6-89d4-fe638debd103-bound-sa-token\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.046622 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.046591 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2049531-ac65-45a6-89d4-fe638debd103-image-registry-private-configuration\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.046664 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.046624 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2049531-ac65-45a6-89d4-fe638debd103-registry-certificates\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.046712 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.046661 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjk4f\" (UniqueName: \"kubernetes.io/projected/f2049531-ac65-45a6-89d4-fe638debd103-kube-api-access-rjk4f\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.046822 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.046795 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2049531-ac65-45a6-89d4-fe638debd103-installation-pull-secrets\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.046901 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.046849 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2049531-ac65-45a6-89d4-fe638debd103-ca-trust-extracted\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.047202 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.047179 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2049531-ac65-45a6-89d4-fe638debd103-ca-trust-extracted\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.047360 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.047339 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2049531-ac65-45a6-89d4-fe638debd103-trusted-ca\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.047425 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.047400 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2049531-ac65-45a6-89d4-fe638debd103-registry-certificates\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.048991 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.048962 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2049531-ac65-45a6-89d4-fe638debd103-registry-tls\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.049081 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.048988 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2049531-ac65-45a6-89d4-fe638debd103-image-registry-private-configuration\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.049406 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.049382 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2049531-ac65-45a6-89d4-fe638debd103-installation-pull-secrets\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.055140 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.055117 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2049531-ac65-45a6-89d4-fe638debd103-bound-sa-token\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.055393 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.055374 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjk4f\" (UniqueName: \"kubernetes.io/projected/f2049531-ac65-45a6-89d4-fe638debd103-kube-api-access-rjk4f\") pod \"image-registry-67748f5cf7-2m5lh\" (UID: \"f2049531-ac65-45a6-89d4-fe638debd103\") " pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.211591 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.211557 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.248521 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.248484 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 17:01:08.250709 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.250690 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65c4ae32-87b7-46f2-8370-510449c1020e-metrics-tls\") pod \"dns-default-dph8q\" (UID: \"65c4ae32-87b7-46f2-8370-510449c1020e\") " pod="openshift-dns/dns-default-dph8q" Jul 15 17:01:08.335822 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.335799 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67748f5cf7-2m5lh"] Jul 15 17:01:08.338185 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:01:08.338154 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2049531_ac65_45a6_89d4_fe638debd103.slice/crio-1c11e613abc44e869807e1d77330e457dde478c10df784781da869e6aa3a6c28 WatchSource:0}: Error finding container 1c11e613abc44e869807e1d77330e457dde478c10df784781da869e6aa3a6c28: Status 404 returned error can't find the container with id 1c11e613abc44e869807e1d77330e457dde478c10df784781da869e6aa3a6c28 Jul 15 17:01:08.406143 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.406124 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lllhc\"" Jul 15 17:01:08.409666 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.409647 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dph8q" Jul 15 17:01:08.413636 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.413605 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" event={"ID":"f2049531-ac65-45a6-89d4-fe638debd103","Type":"ContainerStarted","Data":"fc04595947200907edda2e2ef84a4e6b492660d9beea075e3fa63eb3d9a17280"} Jul 15 17:01:08.413741 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.413663 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" event={"ID":"f2049531-ac65-45a6-89d4-fe638debd103","Type":"ContainerStarted","Data":"1c11e613abc44e869807e1d77330e457dde478c10df784781da869e6aa3a6c28"} Jul 15 17:01:08.413783 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.413732 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:08.445770 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.445724 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" podStartSLOduration=1.445708573 podStartE2EDuration="1.445708573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:01:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:08.444549885 +0000 UTC m=+284.493547077" watchObservedRunningTime="2026-07-15 17:01:08.445708573 +0000 UTC m=+284.494705763" Jul 15 17:01:08.541152 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:08.541050 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dph8q"] Jul 15 17:01:08.543484 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:01:08.543454 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65c4ae32_87b7_46f2_8370_510449c1020e.slice/crio-aae079a0c28e9be21f2b0f5422d7434f7f068494f57fdd10c5ae0f14e64a07e8 WatchSource:0}: Error finding container aae079a0c28e9be21f2b0f5422d7434f7f068494f57fdd10c5ae0f14e64a07e8: Status 404 returned error can't find the container with id aae079a0c28e9be21f2b0f5422d7434f7f068494f57fdd10c5ae0f14e64a07e8 Jul 15 17:01:09.418682 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:09.418643 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dph8q" event={"ID":"65c4ae32-87b7-46f2-8370-510449c1020e","Type":"ContainerStarted","Data":"aae079a0c28e9be21f2b0f5422d7434f7f068494f57fdd10c5ae0f14e64a07e8"} Jul 15 17:01:09.861272 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:09.861197 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:01:09.861272 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:09.861241 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:01:09.861956 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:09.861928 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-service-ca-bundle\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:01:09.863852 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:09.863819 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afcb1c98-744f-4fbe-8396-3217c0d7f2ba-metrics-certs\") pod \"router-default-68ddcc8458-j29rh\" (UID: \"afcb1c98-744f-4fbe-8396-3217c0d7f2ba\") " pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:01:10.071011 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:10.070967 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:01:10.231435 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:10.231413 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68ddcc8458-j29rh"] Jul 15 17:01:10.234827 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:01:10.234790 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafcb1c98_744f_4fbe_8396_3217c0d7f2ba.slice/crio-5660c0898f2da30baaf4cba316ed4be682a439d5969d75e339c7decc18eb38ad WatchSource:0}: Error finding container 5660c0898f2da30baaf4cba316ed4be682a439d5969d75e339c7decc18eb38ad: Status 404 returned error can't find the container with id 5660c0898f2da30baaf4cba316ed4be682a439d5969d75e339c7decc18eb38ad Jul 15 17:01:10.426983 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:10.426944 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68ddcc8458-j29rh" event={"ID":"afcb1c98-744f-4fbe-8396-3217c0d7f2ba","Type":"ContainerStarted","Data":"81acaf3357d818650fdbda72f8a147d2cc61793a19cd869db9ece69620b02d3f"} Jul 15 17:01:10.427449 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:10.427020 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68ddcc8458-j29rh" event={"ID":"afcb1c98-744f-4fbe-8396-3217c0d7f2ba","Type":"ContainerStarted","Data":"5660c0898f2da30baaf4cba316ed4be682a439d5969d75e339c7decc18eb38ad"} Jul 15 17:01:10.428628 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:10.428602 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dph8q" event={"ID":"65c4ae32-87b7-46f2-8370-510449c1020e","Type":"ContainerStarted","Data":"215eeac9f1c09fefe0a47fa688581ff2e1f0a8d0e2cc189b346a671858fdde01"} Jul 15 17:01:10.428729 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:10.428650 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dph8q" event={"ID":"65c4ae32-87b7-46f2-8370-510449c1020e","Type":"ContainerStarted","Data":"4f64cacb65c0b948747192f3bb0b2575f8bac042611a50c71062bdc72af41aa6"} Jul 15 17:01:10.428729 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:10.428688 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dph8q" Jul 15 17:01:10.452396 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:10.452346 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-68ddcc8458-j29rh" podStartSLOduration=33.452332282 podStartE2EDuration="33.452332282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:10.451029222 +0000 UTC m=+286.500026412" watchObservedRunningTime="2026-07-15 17:01:10.452332282 +0000 UTC m=+286.501329472" Jul 15 17:01:10.470198 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:10.470155 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dph8q" podStartSLOduration=250.882839232 podStartE2EDuration="4m12.470139851s" totalImagesPullingTime="1.587300619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:10.469431173 +0000 UTC m=+286.518428365" watchObservedRunningTime="2026-07-15 17:01:10.470139851 +0000 UTC m=+286.519137079" Jul 15 17:01:11.071424 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:11.071388 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:01:11.073833 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:11.073807 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:01:11.432107 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:11.432082 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:01:11.433219 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:11.433196 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-68ddcc8458-j29rh" Jul 15 17:01:14.899957 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:14.899915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:01:14.902142 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:14.902118 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c38aa01-993a-4d62-aeaa-87020199fca1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vz9xc\" (UID: \"5c38aa01-993a-4d62-aeaa-87020199fca1\") " pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:01:15.125962 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:15.125934 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vz9xc" Jul 15 17:01:15.253612 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:15.253542 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vz9xc"] Jul 15 17:01:15.257432 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:01:15.257400 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c38aa01_993a_4d62_aeaa_87020199fca1.slice/crio-81d8e98521a386a4180b80df787cb6132b05f145e2cb2eeb422ef25d9a33c3fd WatchSource:0}: Error finding container 81d8e98521a386a4180b80df787cb6132b05f145e2cb2eeb422ef25d9a33c3fd: Status 404 returned error can't find the container with id 81d8e98521a386a4180b80df787cb6132b05f145e2cb2eeb422ef25d9a33c3fd Jul 15 17:01:15.446496 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:15.446403 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vz9xc" event={"ID":"5c38aa01-993a-4d62-aeaa-87020199fca1","Type":"ContainerStarted","Data":"81d8e98521a386a4180b80df787cb6132b05f145e2cb2eeb422ef25d9a33c3fd"} Jul 15 17:01:15.562352 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:01:15.562307 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mbqkc" podUID="4f43f08e-aa70-42d2-9207-93156d956d7f" Jul 15 17:01:17.862035 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:17.861993 2562 patch_prober.go:28] interesting pod/image-registry-7984794bd6-p4vld container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 17:01:17.862496 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:17.862064 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7984794bd6-p4vld" podUID="5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 17:01:20.433471 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:20.433441 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dph8q" Jul 15 17:01:20.465552 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:20.465521 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vz9xc" event={"ID":"5c38aa01-993a-4d62-aeaa-87020199fca1","Type":"ContainerStarted","Data":"dea24eb1d8c9f50ca8d8e467046fa075aa40e9a78f8b3ec4a320f42aa8b24a15"} Jul 15 17:01:23.479220 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:23.479182 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vz9xc" event={"ID":"5c38aa01-993a-4d62-aeaa-87020199fca1","Type":"ContainerStarted","Data":"c79755804ac774b1245873cc76d5ae0f15fd3ba8c0acf7052f507109458fe7b9"} Jul 15 17:01:24.353398 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:24.353367 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:01:24.358410 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:24.358383 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:01:24.362132 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:24.362112 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:01:24.363808 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:24.363790 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 17:01:24.366779 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:24.366763 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:01:27.860956 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:27.860922 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:01:28.217250 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:28.217216 2562 patch_prober.go:28] interesting pod/image-registry-67748f5cf7-2m5lh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 17:01:28.217515 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:28.217275 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" podUID="f2049531-ac65-45a6-89d4-fe638debd103" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 17:01:28.561226 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:28.561140 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 17:01:29.424316 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:29.424269 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67748f5cf7-2m5lh" Jul 15 17:01:30.505768 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:30.505730 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vz9xc" event={"ID":"5c38aa01-993a-4d62-aeaa-87020199fca1","Type":"ContainerStarted","Data":"6ce9e5ec1c48f0500504cb1948af6364636a0ef651f79ab1aa30d335b1719c4e"} Jul 15 17:01:30.523553 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:30.523503 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vz9xc" podStartSLOduration=33.787389717 podStartE2EDuration="48.523490137s" totalImagesPullingTime="14.73610042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 17:00:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:30.522445789 +0000 UTC m=+306.571442979" watchObservedRunningTime="2026-07-15 17:01:30.523490137 +0000 UTC m=+306.572487327" Jul 15 17:01:32.875308 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:32.875247 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7984794bd6-p4vld" podUID="5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c" containerName="registry" containerID="cri-o://ed7d4af041c3298128126c6d9e03be39a94ddd35d167020e0b0da198e96fcfe4" gracePeriod=30 Jul 15 17:01:33.108390 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.108365 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:01:33.257810 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.257781 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-ca-trust-extracted\") pod \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " Jul 15 17:01:33.257996 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.257851 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-bound-sa-token\") pod \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " Jul 15 17:01:33.257996 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.257875 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lmt2\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-kube-api-access-4lmt2\") pod \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " Jul 15 17:01:33.257996 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.257897 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-installation-pull-secrets\") pod \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " Jul 15 17:01:33.257996 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.257925 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-image-registry-private-configuration\") pod \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " Jul 15 17:01:33.258260 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.258062 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-certificates\") pod \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " Jul 15 17:01:33.258260 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.258094 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls\") pod \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " Jul 15 17:01:33.258260 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.258122 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-trusted-ca\") pod \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\" (UID: \"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c\") " Jul 15 17:01:33.258718 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.258693 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c" (UID: "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:01:33.258807 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.258779 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c" (UID: "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:01:33.260567 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.260520 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c" (UID: "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:01:33.260673 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.260605 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c" (UID: "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:01:33.260673 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.260645 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c" (UID: "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:01:33.260673 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.260655 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-kube-api-access-4lmt2" (OuterVolumeSpecName: "kube-api-access-4lmt2") pod "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c" (UID: "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c"). InnerVolumeSpecName "kube-api-access-4lmt2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:01:33.260788 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.260679 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c" (UID: "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:01:33.267127 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.267106 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c" (UID: "5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:01:33.359458 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.359428 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-trusted-ca\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:01:33.359458 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.359455 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-ca-trust-extracted\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:01:33.359458 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.359465 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-bound-sa-token\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:01:33.359648 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.359474 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lmt2\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-kube-api-access-4lmt2\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:01:33.359648 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.359484 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-installation-pull-secrets\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:01:33.359648 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.359493 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-image-registry-private-configuration\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:01:33.359648 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.359502 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-certificates\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:01:33.359648 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.359510 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c-registry-tls\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:01:33.518475 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.518399 2562 generic.go:356] "Generic (PLEG): container finished" podID="5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c" containerID="ed7d4af041c3298128126c6d9e03be39a94ddd35d167020e0b0da198e96fcfe4" exitCode=0 Jul 15 17:01:33.518475 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.518449 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7984794bd6-p4vld" Jul 15 17:01:33.518475 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.518464 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7984794bd6-p4vld" event={"ID":"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c","Type":"ContainerDied","Data":"ed7d4af041c3298128126c6d9e03be39a94ddd35d167020e0b0da198e96fcfe4"} Jul 15 17:01:33.518643 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.518514 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7984794bd6-p4vld" event={"ID":"5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c","Type":"ContainerDied","Data":"ac4f93c5ac7c5b3cf87a1153c0ebe7a1c03b8240bafd9d7d2b942012bf721a76"} Jul 15 17:01:33.518643 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.518542 2562 scope.go:117] "RemoveContainer" containerID="ed7d4af041c3298128126c6d9e03be39a94ddd35d167020e0b0da198e96fcfe4" Jul 15 17:01:33.528803 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.528709 2562 scope.go:117] "RemoveContainer" containerID="ed7d4af041c3298128126c6d9e03be39a94ddd35d167020e0b0da198e96fcfe4" Jul 15 17:01:33.529450 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:01:33.529385 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed7d4af041c3298128126c6d9e03be39a94ddd35d167020e0b0da198e96fcfe4\": container with ID starting with ed7d4af041c3298128126c6d9e03be39a94ddd35d167020e0b0da198e96fcfe4 not found: ID does not exist" containerID="ed7d4af041c3298128126c6d9e03be39a94ddd35d167020e0b0da198e96fcfe4" Jul 15 17:01:33.529450 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.529420 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed7d4af041c3298128126c6d9e03be39a94ddd35d167020e0b0da198e96fcfe4"} err="failed to get container status \"ed7d4af041c3298128126c6d9e03be39a94ddd35d167020e0b0da198e96fcfe4\": rpc error: code = NotFound desc = could not find container \"ed7d4af041c3298128126c6d9e03be39a94ddd35d167020e0b0da198e96fcfe4\": container with ID starting with ed7d4af041c3298128126c6d9e03be39a94ddd35d167020e0b0da198e96fcfe4 not found: ID does not exist" Jul 15 17:01:33.541622 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.541597 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7984794bd6-p4vld"] Jul 15 17:01:33.545278 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:33.545257 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7984794bd6-p4vld"] Jul 15 17:01:34.564623 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:34.564592 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c" path="/var/lib/kubelet/pods/5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c/volumes" Jul 15 17:01:36.798676 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.798641 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq"] Jul 15 17:01:36.799200 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.798950 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c" containerName="registry" Jul 15 17:01:36.799200 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.798967 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c" containerName="registry" Jul 15 17:01:36.799200 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.799042 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ed58ef1-012e-47e7-b8c4-9625f4dd6e1c" containerName="registry" Jul 15 17:01:36.802815 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.802795 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" Jul 15 17:01:36.805406 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.805379 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 15 17:01:36.805688 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.805667 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 17:01:36.806410 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.806391 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 15 17:01:36.806512 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.806451 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 17:01:36.806715 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.806615 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 17:01:36.806715 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.806618 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bdgsg\"" Jul 15 17:01:36.816616 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.814090 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq"] Jul 15 17:01:36.816616 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.815805 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9cbdd"] Jul 15 17:01:36.820009 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.819278 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-w8mhz"] Jul 15 17:01:36.820009 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.819408 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:36.821616 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.821599 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:36.822192 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.821740 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 17:01:36.822339 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.821791 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 17:01:36.822455 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.821852 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zdfnb\"" Jul 15 17:01:36.822517 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.821946 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 17:01:36.824112 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.824092 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-xpsrw\"" Jul 15 17:01:36.824257 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.824239 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 15 17:01:36.824417 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.824165 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 15 17:01:36.824550 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.824128 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 15 17:01:36.834417 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.834397 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-w8mhz"] Jul 15 17:01:36.985137 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985101 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/171694b4-df85-4c8a-9449-88aad1eeca61-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:36.985137 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985142 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b560a914-ac4b-4113-a075-1ac85ec8f68a-node-exporter-tls\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:36.985373 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985164 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwswd\" (UniqueName: \"kubernetes.io/projected/b560a914-ac4b-4113-a075-1ac85ec8f68a-kube-api-access-mwswd\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:36.985373 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985199 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r6vj\" (UniqueName: \"kubernetes.io/projected/520590e3-2315-46d9-ab1a-6cfdfc36fcea-kube-api-access-6r6vj\") pod \"openshift-state-metrics-84bd68b896-9h5nq\" (UID: \"520590e3-2315-46d9-ab1a-6cfdfc36fcea\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" Jul 15 17:01:36.985373 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985246 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/520590e3-2315-46d9-ab1a-6cfdfc36fcea-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-9h5nq\" (UID: \"520590e3-2315-46d9-ab1a-6cfdfc36fcea\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" Jul 15 17:01:36.985466 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985373 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b560a914-ac4b-4113-a075-1ac85ec8f68a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:36.985466 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985424 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b560a914-ac4b-4113-a075-1ac85ec8f68a-root\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:36.985466 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985448 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b560a914-ac4b-4113-a075-1ac85ec8f68a-sys\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:36.985552 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985473 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/171694b4-df85-4c8a-9449-88aad1eeca61-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:36.985552 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985546 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b560a914-ac4b-4113-a075-1ac85ec8f68a-node-exporter-accelerators-collector-config\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:36.985613 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985570 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/520590e3-2315-46d9-ab1a-6cfdfc36fcea-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-9h5nq\" (UID: \"520590e3-2315-46d9-ab1a-6cfdfc36fcea\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" Jul 15 17:01:36.985613 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985604 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/171694b4-df85-4c8a-9449-88aad1eeca61-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:36.985678 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985626 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b560a914-ac4b-4113-a075-1ac85ec8f68a-node-exporter-textfile\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:36.985678 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985659 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/171694b4-df85-4c8a-9449-88aad1eeca61-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:36.985746 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985683 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/171694b4-df85-4c8a-9449-88aad1eeca61-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:36.985746 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985713 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b560a914-ac4b-4113-a075-1ac85ec8f68a-metrics-client-ca\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:36.985746 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985741 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nc7x\" (UniqueName: \"kubernetes.io/projected/171694b4-df85-4c8a-9449-88aad1eeca61-kube-api-access-4nc7x\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:36.985833 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985763 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b560a914-ac4b-4113-a075-1ac85ec8f68a-node-exporter-wtmp\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:36.985833 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:36.985787 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/520590e3-2315-46d9-ab1a-6cfdfc36fcea-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-9h5nq\" (UID: \"520590e3-2315-46d9-ab1a-6cfdfc36fcea\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" Jul 15 17:01:37.087003 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.086911 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r6vj\" (UniqueName: \"kubernetes.io/projected/520590e3-2315-46d9-ab1a-6cfdfc36fcea-kube-api-access-6r6vj\") pod \"openshift-state-metrics-84bd68b896-9h5nq\" (UID: \"520590e3-2315-46d9-ab1a-6cfdfc36fcea\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" Jul 15 17:01:37.087003 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.086950 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/520590e3-2315-46d9-ab1a-6cfdfc36fcea-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-9h5nq\" (UID: \"520590e3-2315-46d9-ab1a-6cfdfc36fcea\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" Jul 15 17:01:37.087003 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.086971 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b560a914-ac4b-4113-a075-1ac85ec8f68a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.087003 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.086994 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b560a914-ac4b-4113-a075-1ac85ec8f68a-root\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.087348 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087017 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b560a914-ac4b-4113-a075-1ac85ec8f68a-sys\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.087348 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087071 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b560a914-ac4b-4113-a075-1ac85ec8f68a-sys\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.087348 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087089 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b560a914-ac4b-4113-a075-1ac85ec8f68a-root\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.087348 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087144 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/171694b4-df85-4c8a-9449-88aad1eeca61-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:37.087348 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087202 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b560a914-ac4b-4113-a075-1ac85ec8f68a-node-exporter-accelerators-collector-config\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.087348 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087231 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/520590e3-2315-46d9-ab1a-6cfdfc36fcea-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-9h5nq\" (UID: \"520590e3-2315-46d9-ab1a-6cfdfc36fcea\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" Jul 15 17:01:37.087348 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087269 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/171694b4-df85-4c8a-9449-88aad1eeca61-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:37.087348 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087308 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b560a914-ac4b-4113-a075-1ac85ec8f68a-node-exporter-textfile\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.087348 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087343 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/171694b4-df85-4c8a-9449-88aad1eeca61-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:37.087777 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087372 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/171694b4-df85-4c8a-9449-88aad1eeca61-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:37.087777 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087396 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b560a914-ac4b-4113-a075-1ac85ec8f68a-metrics-client-ca\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.087777 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087427 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nc7x\" (UniqueName: \"kubernetes.io/projected/171694b4-df85-4c8a-9449-88aad1eeca61-kube-api-access-4nc7x\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:37.087777 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087452 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b560a914-ac4b-4113-a075-1ac85ec8f68a-node-exporter-wtmp\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.087777 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087478 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/520590e3-2315-46d9-ab1a-6cfdfc36fcea-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-9h5nq\" (UID: \"520590e3-2315-46d9-ab1a-6cfdfc36fcea\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" Jul 15 17:01:37.087777 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087525 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/171694b4-df85-4c8a-9449-88aad1eeca61-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:37.087777 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b560a914-ac4b-4113-a075-1ac85ec8f68a-node-exporter-tls\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.087777 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087573 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwswd\" (UniqueName: \"kubernetes.io/projected/b560a914-ac4b-4113-a075-1ac85ec8f68a-kube-api-access-mwswd\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.087777 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087700 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b560a914-ac4b-4113-a075-1ac85ec8f68a-node-exporter-textfile\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.088203 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087803 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/520590e3-2315-46d9-ab1a-6cfdfc36fcea-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-9h5nq\" (UID: \"520590e3-2315-46d9-ab1a-6cfdfc36fcea\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" Jul 15 17:01:37.088203 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087831 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b560a914-ac4b-4113-a075-1ac85ec8f68a-node-exporter-accelerators-collector-config\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.088203 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:01:37.087907 2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 15 17:01:37.088203 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087946 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/171694b4-df85-4c8a-9449-88aad1eeca61-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:37.088203 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.087962 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b560a914-ac4b-4113-a075-1ac85ec8f68a-node-exporter-wtmp\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.088203 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:01:37.087969 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/171694b4-df85-4c8a-9449-88aad1eeca61-kube-state-metrics-tls podName:171694b4-df85-4c8a-9449-88aad1eeca61 nodeName:}" failed. No retries permitted until 2026-07-15 17:01:37.587952609 +0000 UTC m=+313.636949786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/171694b4-df85-4c8a-9449-88aad1eeca61-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-w8mhz" (UID: "171694b4-df85-4c8a-9449-88aad1eeca61") : secret "kube-state-metrics-tls" not found Jul 15 17:01:37.088514 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.088278 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/171694b4-df85-4c8a-9449-88aad1eeca61-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:37.088566 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.088541 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/171694b4-df85-4c8a-9449-88aad1eeca61-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:37.088968 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.088932 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b560a914-ac4b-4113-a075-1ac85ec8f68a-metrics-client-ca\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.090474 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.090445 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/171694b4-df85-4c8a-9449-88aad1eeca61-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:37.090750 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.090730 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b560a914-ac4b-4113-a075-1ac85ec8f68a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.090959 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.090936 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/520590e3-2315-46d9-ab1a-6cfdfc36fcea-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-9h5nq\" (UID: \"520590e3-2315-46d9-ab1a-6cfdfc36fcea\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" Jul 15 17:01:37.091127 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.091109 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b560a914-ac4b-4113-a075-1ac85ec8f68a-node-exporter-tls\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.091309 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.091271 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/520590e3-2315-46d9-ab1a-6cfdfc36fcea-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-9h5nq\" (UID: \"520590e3-2315-46d9-ab1a-6cfdfc36fcea\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" Jul 15 17:01:37.095821 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.095800 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwswd\" (UniqueName: \"kubernetes.io/projected/b560a914-ac4b-4113-a075-1ac85ec8f68a-kube-api-access-mwswd\") pod \"node-exporter-9cbdd\" (UID: \"b560a914-ac4b-4113-a075-1ac85ec8f68a\") " pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.096237 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.096217 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r6vj\" (UniqueName: \"kubernetes.io/projected/520590e3-2315-46d9-ab1a-6cfdfc36fcea-kube-api-access-6r6vj\") pod \"openshift-state-metrics-84bd68b896-9h5nq\" (UID: \"520590e3-2315-46d9-ab1a-6cfdfc36fcea\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" Jul 15 17:01:37.096605 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.096585 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nc7x\" (UniqueName: \"kubernetes.io/projected/171694b4-df85-4c8a-9449-88aad1eeca61-kube-api-access-4nc7x\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:37.114474 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.114452 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" Jul 15 17:01:37.133537 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.133505 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9cbdd" Jul 15 17:01:37.144874 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:01:37.144846 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb560a914_ac4b_4113_a075_1ac85ec8f68a.slice/crio-13d6a39a59fe49ea8414cd3eecc6d80063098cedb46ea0aadc36ce0fa0c50192 WatchSource:0}: Error finding container 13d6a39a59fe49ea8414cd3eecc6d80063098cedb46ea0aadc36ce0fa0c50192: Status 404 returned error can't find the container with id 13d6a39a59fe49ea8414cd3eecc6d80063098cedb46ea0aadc36ce0fa0c50192 Jul 15 17:01:37.147069 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.147051 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 17:01:37.240405 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.240373 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq"] Jul 15 17:01:37.243161 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:01:37.243135 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod520590e3_2315_46d9_ab1a_6cfdfc36fcea.slice/crio-6d3bd4f3390fafd4922f52334117e00b3412ad1bf12233acf114acfc143b7107 WatchSource:0}: Error finding container 6d3bd4f3390fafd4922f52334117e00b3412ad1bf12233acf114acfc143b7107: Status 404 returned error can't find the container with id 6d3bd4f3390fafd4922f52334117e00b3412ad1bf12233acf114acfc143b7107 Jul 15 17:01:37.535392 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.535354 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" event={"ID":"520590e3-2315-46d9-ab1a-6cfdfc36fcea","Type":"ContainerStarted","Data":"0d4554e099f286ea2ffe4c61406c14363ed5faa4c457bcd53a5de222fe8ae6b9"} Jul 15 17:01:37.535561 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.535437 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" event={"ID":"520590e3-2315-46d9-ab1a-6cfdfc36fcea","Type":"ContainerStarted","Data":"abc92ed5c0d92c464f38849cb064ad19e3f70907b4f3a7a88e902fc7029ad4ca"} Jul 15 17:01:37.535561 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.535462 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" event={"ID":"520590e3-2315-46d9-ab1a-6cfdfc36fcea","Type":"ContainerStarted","Data":"6d3bd4f3390fafd4922f52334117e00b3412ad1bf12233acf114acfc143b7107"} Jul 15 17:01:37.536350 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.536327 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9cbdd" event={"ID":"b560a914-ac4b-4113-a075-1ac85ec8f68a","Type":"ContainerStarted","Data":"13d6a39a59fe49ea8414cd3eecc6d80063098cedb46ea0aadc36ce0fa0c50192"} Jul 15 17:01:37.591117 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.591050 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/171694b4-df85-4c8a-9449-88aad1eeca61-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:37.593346 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.593327 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/171694b4-df85-4c8a-9449-88aad1eeca61-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-w8mhz\" (UID: \"171694b4-df85-4c8a-9449-88aad1eeca61\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:37.739750 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.739694 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" Jul 15 17:01:37.887599 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:37.887513 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-w8mhz"] Jul 15 17:01:38.082299 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:01:38.082265 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod171694b4_df85_4c8a_9449_88aad1eeca61.slice/crio-ed648446c024494483dc5e1c765f18d6ba77570ed681c698aef6a2657977c747 WatchSource:0}: Error finding container ed648446c024494483dc5e1c765f18d6ba77570ed681c698aef6a2657977c747: Status 404 returned error can't find the container with id ed648446c024494483dc5e1c765f18d6ba77570ed681c698aef6a2657977c747 Jul 15 17:01:38.541749 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.541714 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" event={"ID":"520590e3-2315-46d9-ab1a-6cfdfc36fcea","Type":"ContainerStarted","Data":"22ee85fea5516a9a81fc370ebcb287cee01dfe08e749d6b72bb4331f708b2f27"} Jul 15 17:01:38.543112 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.543086 2562 generic.go:356] "Generic (PLEG): container finished" podID="b560a914-ac4b-4113-a075-1ac85ec8f68a" containerID="4b4e3aa9e00e8434dda8f06829c5323c7b2838cd52844274e56bd92fe6ff50cb" exitCode=0 Jul 15 17:01:38.543246 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.543160 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9cbdd" event={"ID":"b560a914-ac4b-4113-a075-1ac85ec8f68a","Type":"ContainerDied","Data":"4b4e3aa9e00e8434dda8f06829c5323c7b2838cd52844274e56bd92fe6ff50cb"} Jul 15 17:01:38.544153 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.544136 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" event={"ID":"171694b4-df85-4c8a-9449-88aad1eeca61","Type":"ContainerStarted","Data":"ed648446c024494483dc5e1c765f18d6ba77570ed681c698aef6a2657977c747"} Jul 15 17:01:38.562678 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.562630 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9h5nq" podStartSLOduration=1.60113645 podStartE2EDuration="2.562613613s" totalImagesPullingTime="961.477163ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:01:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:38.56236653 +0000 UTC m=+314.611363720" watchObservedRunningTime="2026-07-15 17:01:38.562613613 +0000 UTC m=+314.611610814" Jul 15 17:01:38.801987 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.801951 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-666f98849c-58htg"] Jul 15 17:01:38.804991 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.804965 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:38.807596 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.807539 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 15 17:01:38.808126 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.807890 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2fqc4ofq6i5e3\"" Jul 15 17:01:38.808126 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.807902 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 15 17:01:38.808126 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.807966 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-vqnzb\"" Jul 15 17:01:38.808126 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.808012 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 15 17:01:38.808126 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.807970 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 15 17:01:38.808126 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.808096 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 15 17:01:38.818284 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.818265 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-666f98849c-58htg"] Jul 15 17:01:38.903183 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.903153 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:38.903628 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.903195 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djwjr\" (UniqueName: \"kubernetes.io/projected/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-kube-api-access-djwjr\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:38.903628 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.903224 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:38.903628 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.903264 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:38.903628 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.903348 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-metrics-client-ca\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:38.903628 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.903394 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:38.903628 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.903429 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-grpc-tls\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:38.903628 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:38.903488 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-thanos-querier-tls\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.004748 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.004715 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-metrics-client-ca\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.004931 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.004770 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.004931 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.004796 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-grpc-tls\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.004931 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.004826 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-thanos-querier-tls\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.004931 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.004866 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.005332 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.005303 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djwjr\" (UniqueName: \"kubernetes.io/projected/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-kube-api-access-djwjr\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.005428 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.005361 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.005495 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.005421 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.005545 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.005523 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-metrics-client-ca\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.007728 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.007705 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.007990 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.007948 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-thanos-querier-tls\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.008107 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.008083 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-grpc-tls\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.008250 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.008230 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.008328 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.008311 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.008455 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.008439 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.014026 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.014005 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djwjr\" (UniqueName: \"kubernetes.io/projected/bb1bfc1d-a8b7-4198-b629-73b6ae489e88-kube-api-access-djwjr\") pod \"thanos-querier-666f98849c-58htg\" (UID: \"bb1bfc1d-a8b7-4198-b629-73b6ae489e88\") " pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.115754 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.115684 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:39.495602 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.495541 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-666f98849c-58htg"] Jul 15 17:01:39.498367 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:01:39.498334 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb1bfc1d_a8b7_4198_b629_73b6ae489e88.slice/crio-d7c8c9cdad2ac20545c585d0742b5149d3d027bd7222acbedc1188d6f2850889 WatchSource:0}: Error finding container d7c8c9cdad2ac20545c585d0742b5149d3d027bd7222acbedc1188d6f2850889: Status 404 returned error can't find the container with id d7c8c9cdad2ac20545c585d0742b5149d3d027bd7222acbedc1188d6f2850889 Jul 15 17:01:39.551074 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.551025 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-666f98849c-58htg" event={"ID":"bb1bfc1d-a8b7-4198-b629-73b6ae489e88","Type":"ContainerStarted","Data":"d7c8c9cdad2ac20545c585d0742b5149d3d027bd7222acbedc1188d6f2850889"} Jul 15 17:01:39.554562 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.553915 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9cbdd" event={"ID":"b560a914-ac4b-4113-a075-1ac85ec8f68a","Type":"ContainerStarted","Data":"953176e88cf7206380c975cd8cfe0e58691f0f07b5c7957c704ba51d6699c87b"} Jul 15 17:01:39.554562 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.553974 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9cbdd" event={"ID":"b560a914-ac4b-4113-a075-1ac85ec8f68a","Type":"ContainerStarted","Data":"136be1d1b08a659ad4094fcebac6be6982d2bc1283809733b0dd4893358153b8"} Jul 15 17:01:39.557242 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.557212 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" event={"ID":"171694b4-df85-4c8a-9449-88aad1eeca61","Type":"ContainerStarted","Data":"726c7702505b2781183d868dbc5fc4e2038de0a5d2c9a82c86146a05e0fd08c9"} Jul 15 17:01:39.557442 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.557413 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" event={"ID":"171694b4-df85-4c8a-9449-88aad1eeca61","Type":"ContainerStarted","Data":"dbf7aff28b6faddec0c17f4c27a7053266d8b4ba2237eef8a1e3f28b18817809"} Jul 15 17:01:39.576470 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:39.575247 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9cbdd" podStartSLOduration=2.347675421 podStartE2EDuration="3.575223673s" totalImagesPullingTime="1.161174717s" totalInitContainerRuntime="66.373535ms" isStatefulPod=true podCreationTimestamp="2026-07-15 17:01:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:39.574573805 +0000 UTC m=+315.623571021" watchObservedRunningTime="2026-07-15 17:01:39.575223673 +0000 UTC m=+315.624220868" Jul 15 17:01:40.564713 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:40.564677 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" event={"ID":"171694b4-df85-4c8a-9449-88aad1eeca61","Type":"ContainerStarted","Data":"f1d3fd62d48d089e09e776589490995a94c620ff653cf836b7f3877c6fb8f600"} Jul 15 17:01:40.585014 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:40.584949 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-w8mhz" podStartSLOduration=3.27208586 podStartE2EDuration="4.58493558s" totalImagesPullingTime="1.31284972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:01:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:40.584720692 +0000 UTC m=+316.633717882" watchObservedRunningTime="2026-07-15 17:01:40.58493558 +0000 UTC m=+316.633932771" Jul 15 17:01:41.119366 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.119326 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t"] Jul 15 17:01:41.121481 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.121460 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.124355 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.124334 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 15 17:01:41.124459 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.124334 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-da82m1re99cii\"" Jul 15 17:01:41.125199 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.125174 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 15 17:01:41.125324 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.125234 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 15 17:01:41.125324 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.125234 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 15 17:01:41.125532 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.125517 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-kpsbw\"" Jul 15 17:01:41.133197 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.133178 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t"] Jul 15 17:01:41.226032 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.225994 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-secret-metrics-server-tls\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.226195 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.226039 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-audit-log\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.226195 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.226107 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-client-ca-bundle\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.226195 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.226151 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-secret-metrics-server-client-certs\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.226335 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.226220 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.226335 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.226258 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-metrics-server-audit-profiles\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.226335 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.226329 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tjzn\" (UniqueName: \"kubernetes.io/projected/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-kube-api-access-9tjzn\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.327141 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.327107 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.327282 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.327153 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-metrics-server-audit-profiles\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.327282 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.327182 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tjzn\" (UniqueName: \"kubernetes.io/projected/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-kube-api-access-9tjzn\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.327282 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.327214 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-secret-metrics-server-tls\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.327282 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.327231 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-audit-log\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.327512 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.327405 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-client-ca-bundle\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.327512 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.327454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-secret-metrics-server-client-certs\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.327639 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.327619 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-audit-log\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.327945 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.327915 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.328167 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.328146 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-metrics-server-audit-profiles\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.330020 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.329973 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-secret-metrics-server-tls\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.330131 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.330105 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-client-ca-bundle\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.330196 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.330156 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-secret-metrics-server-client-certs\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.337382 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.337342 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tjzn\" (UniqueName: \"kubernetes.io/projected/b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b-kube-api-access-9tjzn\") pod \"metrics-server-56bbd4b4f6-ckt7t\" (UID: \"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b\") " pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.430985 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.430935 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:01:41.571466 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:41.571433 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t"] Jul 15 17:01:41.576733 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:01:41.576696 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e8d0b3_8cdf_4f67_9317_b2ac533f3d4b.slice/crio-509ade3dcc0c53b80c1938fdb930395fba8950e46956aa211238d3c7a5e66979 WatchSource:0}: Error finding container 509ade3dcc0c53b80c1938fdb930395fba8950e46956aa211238d3c7a5e66979: Status 404 returned error can't find the container with id 509ade3dcc0c53b80c1938fdb930395fba8950e46956aa211238d3c7a5e66979 Jul 15 17:01:42.572950 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:42.572908 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" event={"ID":"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b","Type":"ContainerStarted","Data":"509ade3dcc0c53b80c1938fdb930395fba8950e46956aa211238d3c7a5e66979"} Jul 15 17:01:42.575406 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:42.575379 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-666f98849c-58htg" event={"ID":"bb1bfc1d-a8b7-4198-b629-73b6ae489e88","Type":"ContainerStarted","Data":"e01493d5654c62babc5d367f93c0c6036786b0fc35a2dfb0401c5c301ce89161"} Jul 15 17:01:42.575520 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:42.575452 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-666f98849c-58htg" event={"ID":"bb1bfc1d-a8b7-4198-b629-73b6ae489e88","Type":"ContainerStarted","Data":"8febd358e43590ba9881eec3dd50aed63d3aa5b6dea3d9f59bd4b5817ff78d3c"} Jul 15 17:01:42.575520 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:42.575479 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-666f98849c-58htg" event={"ID":"bb1bfc1d-a8b7-4198-b629-73b6ae489e88","Type":"ContainerStarted","Data":"b1ccbe2a5add6a83abde7413a4cfc9205674dd71b96ee4007e1a899dab39ace4"} Jul 15 17:01:43.187550 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.187521 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 17:01:43.191329 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.191306 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.194265 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.194240 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 15 17:01:43.194441 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.194398 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 15 17:01:43.194555 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.194455 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ak9cp0cefmnqb\"" Jul 15 17:01:43.194907 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.194880 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 15 17:01:43.195004 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.194971 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 15 17:01:43.195066 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.195005 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 15 17:01:43.195766 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.195444 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 15 17:01:43.195766 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.195677 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 15 17:01:43.195898 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.195821 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 15 17:01:43.195898 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.195839 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 15 17:01:43.195898 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.195866 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 15 17:01:43.196102 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.195820 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 15 17:01:43.196102 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.195959 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7qccl\"" Jul 15 17:01:43.196102 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.195967 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 15 17:01:43.196506 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.196462 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 15 17:01:43.197940 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.197921 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 15 17:01:43.215773 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.212867 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 17:01:43.244518 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.244477 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.244518 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.244521 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.244710 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.244541 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.244710 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.244564 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.244710 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.244634 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.244710 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.244669 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvskn\" (UniqueName: \"kubernetes.io/projected/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-kube-api-access-wvskn\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.244869 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.244711 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.244869 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.244737 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.244869 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.244790 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.244869 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.244819 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.244869 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.244840 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.245094 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.244868 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.245094 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.244894 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.245094 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.244942 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.245094 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.244983 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.245094 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.245010 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.245094 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.245036 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-config\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.245094 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.245080 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.245395 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.245108 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.245395 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.245129 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.346468 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.346432 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.346468 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.346474 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.346826 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.346502 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.346826 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.346530 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.346826 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.346658 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.346826 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.346755 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-config\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.346826 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.346787 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.346826 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.346825 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.347214 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.346849 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.347214 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.346887 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.347214 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.346915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.347214 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.346944 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.347214 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.346971 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.347214 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.346971 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.347214 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.347053 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.347214 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.347084 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvskn\" (UniqueName: \"kubernetes.io/projected/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-kube-api-access-wvskn\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.347214 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.347134 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.347214 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.347157 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.347214 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.347211 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.347765 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.347238 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.347765 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.347276 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.347765 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.347556 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.347765 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.347575 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.347988 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.347887 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.348851 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.348719 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.349479 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.349314 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.350717 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.349882 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-config-out\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.350717 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.350223 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.350717 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.350668 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.350717 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.350703 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-config\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.350947 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.350832 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.351647 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.351618 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.352189 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.352164 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.352593 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.352430 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.352593 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.352511 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-web-config\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.352920 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.352895 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.353416 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.353399 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.353614 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.353598 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.353810 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.353777 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.356507 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.356489 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvskn\" (UniqueName: \"kubernetes.io/projected/d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe-kube-api-access-wvskn\") pod \"prometheus-k8s-0\" (UID: \"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.501932 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.501897 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:43.582805 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.582775 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-666f98849c-58htg" event={"ID":"bb1bfc1d-a8b7-4198-b629-73b6ae489e88","Type":"ContainerStarted","Data":"c16f79debfd4556591006157481e5af806fe7657bed195a5c3a17b9013f858cf"} Jul 15 17:01:43.583134 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.582839 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-666f98849c-58htg" event={"ID":"bb1bfc1d-a8b7-4198-b629-73b6ae489e88","Type":"ContainerStarted","Data":"bdff0a43b80d22036aadfd6c06c7f50b20db629b2048841abccbadf931d6b011"} Jul 15 17:01:43.660237 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:43.660205 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 17:01:43.663612 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:01:43.663580 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1f02325_67bf_4b5b_9e7c_a5f1be27a9fe.slice/crio-236fe45ad6770e04acfad1b5cce7ac6f061a52dd1912a016279e5c29dff5b5ef WatchSource:0}: Error finding container 236fe45ad6770e04acfad1b5cce7ac6f061a52dd1912a016279e5c29dff5b5ef: Status 404 returned error can't find the container with id 236fe45ad6770e04acfad1b5cce7ac6f061a52dd1912a016279e5c29dff5b5ef Jul 15 17:01:44.589162 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:44.588819 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe","Type":"ContainerStarted","Data":"236fe45ad6770e04acfad1b5cce7ac6f061a52dd1912a016279e5c29dff5b5ef"} Jul 15 17:01:44.590732 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:44.590703 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" event={"ID":"b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b","Type":"ContainerStarted","Data":"2800c998e1a9b5c5242ec4ffb25398e204b309a5bf7434b9ff5b0a5876852125"} Jul 15 17:01:44.594093 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:44.594066 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-666f98849c-58htg" event={"ID":"bb1bfc1d-a8b7-4198-b629-73b6ae489e88","Type":"ContainerStarted","Data":"64ba3d53aa3a83997f6012924818bcc1a306ed8441316c8676468d237e2010f8"} Jul 15 17:01:44.594255 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:44.594225 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:44.610468 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:44.610415 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" podStartSLOduration=0.895235134 podStartE2EDuration="3.610398653s" totalImagesPullingTime="2.715163519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:44.608867201 +0000 UTC m=+320.657864393" watchObservedRunningTime="2026-07-15 17:01:44.610398653 +0000 UTC m=+320.659395844" Jul 15 17:01:44.631640 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:44.631596 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-666f98849c-58htg" podStartSLOduration=2.946682588 podStartE2EDuration="6.631581174s" totalImagesPullingTime="3.684898586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:01:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:44.631349178 +0000 UTC m=+320.680346372" watchObservedRunningTime="2026-07-15 17:01:44.631581174 +0000 UTC m=+320.680578365" Jul 15 17:01:45.599226 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:45.599190 2562 generic.go:356] "Generic (PLEG): container finished" podID="d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe" containerID="149627baf8d9587bc108458971c630d0f0bfd902d3e0dd458f57ba8ed8fdae71" exitCode=0 Jul 15 17:01:45.599724 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:45.599281 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe","Type":"ContainerDied","Data":"149627baf8d9587bc108458971c630d0f0bfd902d3e0dd458f57ba8ed8fdae71"} Jul 15 17:01:49.619625 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:49.619534 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe","Type":"ContainerStarted","Data":"d1e942dc73f2dc6a791cb20ef17c8e60df05de7e7ae702417293e550231d3c29"} Jul 15 17:01:49.619625 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:49.619609 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe","Type":"ContainerStarted","Data":"4fdee0a80e1630e9c2c2b70e717ef3153b762876f79db80875dadb3612647e59"} Jul 15 17:01:49.620109 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:49.619651 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe","Type":"ContainerStarted","Data":"550cc1a24625ac127ac3ee4c1cc7e7b2065d2b5c4e21c0b090ab245bc9e60e5b"} Jul 15 17:01:49.620109 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:49.619685 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe","Type":"ContainerStarted","Data":"672053ecf4233c22b0644f20f2945629bd184f9a11f16903ebe2da4ff13a253f"} Jul 15 17:01:49.620109 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:49.619722 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe","Type":"ContainerStarted","Data":"0090a3e327a084a7947e265162465de4f777b488bb6b29b180644011c72e557d"} Jul 15 17:01:49.620109 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:49.619771 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe","Type":"ContainerStarted","Data":"1efeac385f4c8480db14adecde6072de85e02140b05cdd8b8bee32afee40ce9e"} Jul 15 17:01:49.649628 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:49.649576 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.090012114 podStartE2EDuration="6.649555663s" totalImagesPullingTime="4.538745303s" totalInitContainerRuntime="20.798246ms" isStatefulPod=false podCreationTimestamp="2026-07-15 17:01:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:01:49.647415447 +0000 UTC m=+325.696412639" watchObservedRunningTime="2026-07-15 17:01:49.649555663 +0000 UTC m=+325.698552853" Jul 15 17:01:50.606593 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:50.606567 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-666f98849c-58htg" Jul 15 17:01:52.631487 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:52.631457 2562 generic.go:356] "Generic (PLEG): container finished" podID="c7ffc33d-460d-437f-baf0-c93e7efc4dff" containerID="9d37ea099ed08830f375887b26466967761a6c407ab0b2d982e0319bddf15a7b" exitCode=0 Jul 15 17:01:52.631859 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:52.631538 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz" event={"ID":"c7ffc33d-460d-437f-baf0-c93e7efc4dff","Type":"ContainerDied","Data":"9d37ea099ed08830f375887b26466967761a6c407ab0b2d982e0319bddf15a7b"} Jul 15 17:01:52.631899 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:52.631864 2562 scope.go:117] "RemoveContainer" containerID="9d37ea099ed08830f375887b26466967761a6c407ab0b2d982e0319bddf15a7b" Jul 15 17:01:53.502495 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:53.502455 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:01:53.637128 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:01:53.637097 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-27gvz" event={"ID":"c7ffc33d-460d-437f-baf0-c93e7efc4dff","Type":"ContainerStarted","Data":"edbd5ea0c6d0a278dda5e703dca2675345d7d5823c9d4b362d55cebb028d5f42"} Jul 15 17:02:01.431459 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:02:01.431427 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:02:01.431861 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:02:01.431509 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:02:21.441711 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:02:21.441612 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:02:21.448244 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:02:21.448220 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-56bbd4b4f6-ckt7t" Jul 15 17:02:37.229923 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:02:37.229881 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs\") pod \"network-metrics-daemon-mbqkc\" (UID: \"4f43f08e-aa70-42d2-9207-93156d956d7f\") " pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 17:02:37.232102 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:02:37.232075 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f43f08e-aa70-42d2-9207-93156d956d7f-metrics-certs\") pod \"network-metrics-daemon-mbqkc\" (UID: \"4f43f08e-aa70-42d2-9207-93156d956d7f\") " pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 17:02:37.264677 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:02:37.264654 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pqk7d\"" Jul 15 17:02:37.272763 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:02:37.272744 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mbqkc" Jul 15 17:02:37.398857 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:02:37.398771 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mbqkc"] Jul 15 17:02:37.401032 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:02:37.401011 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f43f08e_aa70_42d2_9207_93156d956d7f.slice/crio-4855657d3bed7ae2534961516b46d12f24ebcff6c6dda2a98b1948cb7574c564 WatchSource:0}: Error finding container 4855657d3bed7ae2534961516b46d12f24ebcff6c6dda2a98b1948cb7574c564: Status 404 returned error can't find the container with id 4855657d3bed7ae2534961516b46d12f24ebcff6c6dda2a98b1948cb7574c564 Jul 15 17:02:37.811544 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:02:37.811502 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mbqkc" event={"ID":"4f43f08e-aa70-42d2-9207-93156d956d7f","Type":"ContainerStarted","Data":"4855657d3bed7ae2534961516b46d12f24ebcff6c6dda2a98b1948cb7574c564"} Jul 15 17:02:38.817341 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:02:38.817305 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mbqkc" event={"ID":"4f43f08e-aa70-42d2-9207-93156d956d7f","Type":"ContainerStarted","Data":"8be238764ba8446c5da9aa3dcec07e18ba347b698800179b1de38b63b05bc268"} Jul 15 17:02:38.817694 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:02:38.817362 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mbqkc" event={"ID":"4f43f08e-aa70-42d2-9207-93156d956d7f","Type":"ContainerStarted","Data":"1a2fa81c07f358081d513768235726c55128f16cfe9f80c489f035e09bcc4ca7"} Jul 15 17:02:38.837285 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:02:38.837242 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mbqkc" podStartSLOduration=374.052671378 podStartE2EDuration="6m14.837226396s" totalImagesPullingTime="784.555018ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:02:38.83703649 +0000 UTC m=+374.886033681" watchObservedRunningTime="2026-07-15 17:02:38.837226396 +0000 UTC m=+374.886223587" Jul 15 17:02:43.502938 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:02:43.502907 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:02:43.518263 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:02:43.518239 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:02:43.852048 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:02:43.851976 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 17:05:51.143819 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.143788 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf"] Jul 15 17:05:51.148138 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.148112 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.150843 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.150819 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 17:05:51.151091 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.151074 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 17:05:51.151150 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.151110 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5xhsk\"" Jul 15 17:05:51.151373 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.151352 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 17:05:51.151426 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.151397 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 17:05:51.163992 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.163972 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf"] Jul 15 17:05:51.262070 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.262036 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a240661c-051d-4ee2-840a-9b6ace94b96a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-klfxf\" (UID: \"a240661c-051d-4ee2-840a-9b6ace94b96a\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.262070 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.262068 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh7wj\" (UniqueName: \"kubernetes.io/projected/a240661c-051d-4ee2-840a-9b6ace94b96a-kube-api-access-lh7wj\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-klfxf\" (UID: \"a240661c-051d-4ee2-840a-9b6ace94b96a\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.262264 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.262100 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a240661c-051d-4ee2-840a-9b6ace94b96a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-klfxf\" (UID: \"a240661c-051d-4ee2-840a-9b6ace94b96a\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.262264 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.262158 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a240661c-051d-4ee2-840a-9b6ace94b96a-manifests\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-klfxf\" (UID: \"a240661c-051d-4ee2-840a-9b6ace94b96a\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.262264 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.262197 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a240661c-051d-4ee2-840a-9b6ace94b96a-tmp\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-klfxf\" (UID: \"a240661c-051d-4ee2-840a-9b6ace94b96a\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.362603 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.362568 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a240661c-051d-4ee2-840a-9b6ace94b96a-manifests\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-klfxf\" (UID: \"a240661c-051d-4ee2-840a-9b6ace94b96a\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.362754 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.362621 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a240661c-051d-4ee2-840a-9b6ace94b96a-tmp\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-klfxf\" (UID: \"a240661c-051d-4ee2-840a-9b6ace94b96a\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.362754 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.362670 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a240661c-051d-4ee2-840a-9b6ace94b96a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-klfxf\" (UID: \"a240661c-051d-4ee2-840a-9b6ace94b96a\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.362754 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.362690 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh7wj\" (UniqueName: \"kubernetes.io/projected/a240661c-051d-4ee2-840a-9b6ace94b96a-kube-api-access-lh7wj\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-klfxf\" (UID: \"a240661c-051d-4ee2-840a-9b6ace94b96a\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.362915 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.362754 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a240661c-051d-4ee2-840a-9b6ace94b96a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-klfxf\" (UID: \"a240661c-051d-4ee2-840a-9b6ace94b96a\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.362964 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.362944 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a240661c-051d-4ee2-840a-9b6ace94b96a-manifests\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-klfxf\" (UID: \"a240661c-051d-4ee2-840a-9b6ace94b96a\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.364931 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.364901 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a240661c-051d-4ee2-840a-9b6ace94b96a-tmp\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-klfxf\" (UID: \"a240661c-051d-4ee2-840a-9b6ace94b96a\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.365033 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.364973 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a240661c-051d-4ee2-840a-9b6ace94b96a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-klfxf\" (UID: \"a240661c-051d-4ee2-840a-9b6ace94b96a\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.365125 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.365110 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a240661c-051d-4ee2-840a-9b6ace94b96a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-klfxf\" (UID: \"a240661c-051d-4ee2-840a-9b6ace94b96a\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.372575 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.372554 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh7wj\" (UniqueName: \"kubernetes.io/projected/a240661c-051d-4ee2-840a-9b6ace94b96a-kube-api-access-lh7wj\") pod \"opendatahub-operator-controller-manager-7b767d9f5c-klfxf\" (UID: \"a240661c-051d-4ee2-840a-9b6ace94b96a\") " pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.458013 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.457985 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:51.627171 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:51.627149 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf"] Jul 15 17:05:52.554964 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:52.554910 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" event={"ID":"a240661c-051d-4ee2-840a-9b6ace94b96a","Type":"ContainerStarted","Data":"e95e989d02f4a2d06154539f3a82d2af765883d0c67407c536955d1897f855bb"} Jul 15 17:05:54.569345 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:54.569279 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" event={"ID":"a240661c-051d-4ee2-840a-9b6ace94b96a","Type":"ContainerStarted","Data":"9a08401e477bce4fc14dc2ba5ca35e516ed0291a6c0a467f314e9f1450912732"} Jul 15 17:05:56.578534 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:56.578499 2562 generic.go:356] "Generic (PLEG): container finished" podID="a240661c-051d-4ee2-840a-9b6ace94b96a" containerID="9a08401e477bce4fc14dc2ba5ca35e516ed0291a6c0a467f314e9f1450912732" exitCode=0 Jul 15 17:05:56.578534 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:56.578538 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" event={"ID":"a240661c-051d-4ee2-840a-9b6ace94b96a","Type":"ContainerDied","Data":"9a08401e477bce4fc14dc2ba5ca35e516ed0291a6c0a467f314e9f1450912732"} Jul 15 17:05:57.584946 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:57.584905 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" event={"ID":"a240661c-051d-4ee2-840a-9b6ace94b96a","Type":"ContainerStarted","Data":"36333f065d308a6e6fbb8231d91dfa63680023da30e6f95f87cdc9f51a9fc44e"} Jul 15 17:05:57.585362 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:57.585158 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:05:57.610504 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:05:57.610448 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" podStartSLOduration=2.310611133 podStartE2EDuration="6.610428132s" totalImagesPullingTime="3.143986245s" totalInitContainerRuntime="1.155830754s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:05:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:05:57.609243415 +0000 UTC m=+573.658240606" watchObservedRunningTime="2026-07-15 17:05:57.610428132 +0000 UTC m=+573.659425324" Jul 15 17:06:08.590622 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:08.590592 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b767d9f5c-klfxf" Jul 15 17:06:11.017206 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.017175 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88"] Jul 15 17:06:11.027740 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.027724 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" Jul 15 17:06:11.031892 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.031872 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 15 17:06:11.033062 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.033023 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88"] Jul 15 17:06:11.035473 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.035453 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 15 17:06:11.035604 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.035587 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 17:06:11.036084 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.036067 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 15 17:06:11.036246 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.036229 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 17:06:11.036395 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.036378 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-zzmcx\"" Jul 15 17:06:11.136938 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.136903 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cacf5e90-db63-4e20-a351-8f5aac61e289-metrics-cert\") pod \"lws-controller-manager-5c485d79f6-zhw88\" (UID: \"cacf5e90-db63-4e20-a351-8f5aac61e289\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" Jul 15 17:06:11.137130 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.136952 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cacf5e90-db63-4e20-a351-8f5aac61e289-manager-config\") pod \"lws-controller-manager-5c485d79f6-zhw88\" (UID: \"cacf5e90-db63-4e20-a351-8f5aac61e289\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" Jul 15 17:06:11.137130 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.137046 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd2ql\" (UniqueName: \"kubernetes.io/projected/cacf5e90-db63-4e20-a351-8f5aac61e289-kube-api-access-rd2ql\") pod \"lws-controller-manager-5c485d79f6-zhw88\" (UID: \"cacf5e90-db63-4e20-a351-8f5aac61e289\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" Jul 15 17:06:11.137130 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.137083 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cacf5e90-db63-4e20-a351-8f5aac61e289-cert\") pod \"lws-controller-manager-5c485d79f6-zhw88\" (UID: \"cacf5e90-db63-4e20-a351-8f5aac61e289\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" Jul 15 17:06:11.238147 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.238115 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cacf5e90-db63-4e20-a351-8f5aac61e289-metrics-cert\") pod \"lws-controller-manager-5c485d79f6-zhw88\" (UID: \"cacf5e90-db63-4e20-a351-8f5aac61e289\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" Jul 15 17:06:11.238285 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.238156 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cacf5e90-db63-4e20-a351-8f5aac61e289-manager-config\") pod \"lws-controller-manager-5c485d79f6-zhw88\" (UID: \"cacf5e90-db63-4e20-a351-8f5aac61e289\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" Jul 15 17:06:11.238285 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.238205 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd2ql\" (UniqueName: \"kubernetes.io/projected/cacf5e90-db63-4e20-a351-8f5aac61e289-kube-api-access-rd2ql\") pod \"lws-controller-manager-5c485d79f6-zhw88\" (UID: \"cacf5e90-db63-4e20-a351-8f5aac61e289\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" Jul 15 17:06:11.238285 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.238231 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cacf5e90-db63-4e20-a351-8f5aac61e289-cert\") pod \"lws-controller-manager-5c485d79f6-zhw88\" (UID: \"cacf5e90-db63-4e20-a351-8f5aac61e289\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" Jul 15 17:06:11.238887 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.238862 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cacf5e90-db63-4e20-a351-8f5aac61e289-manager-config\") pod \"lws-controller-manager-5c485d79f6-zhw88\" (UID: \"cacf5e90-db63-4e20-a351-8f5aac61e289\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" Jul 15 17:06:11.240607 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.240587 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cacf5e90-db63-4e20-a351-8f5aac61e289-cert\") pod \"lws-controller-manager-5c485d79f6-zhw88\" (UID: \"cacf5e90-db63-4e20-a351-8f5aac61e289\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" Jul 15 17:06:11.240686 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.240633 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cacf5e90-db63-4e20-a351-8f5aac61e289-metrics-cert\") pod \"lws-controller-manager-5c485d79f6-zhw88\" (UID: \"cacf5e90-db63-4e20-a351-8f5aac61e289\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" Jul 15 17:06:11.252790 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.252769 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd2ql\" (UniqueName: \"kubernetes.io/projected/cacf5e90-db63-4e20-a351-8f5aac61e289-kube-api-access-rd2ql\") pod \"lws-controller-manager-5c485d79f6-zhw88\" (UID: \"cacf5e90-db63-4e20-a351-8f5aac61e289\") " pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" Jul 15 17:06:11.338097 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.338016 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" Jul 15 17:06:11.472765 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.472730 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88"] Jul 15 17:06:11.479709 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:06:11.479679 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcacf5e90_db63_4e20_a351_8f5aac61e289.slice/crio-f423296e39accbd29a16e1e27d1d424a871110ba12c37f6d2b4f7c6c111da34e WatchSource:0}: Error finding container f423296e39accbd29a16e1e27d1d424a871110ba12c37f6d2b4f7c6c111da34e: Status 404 returned error can't find the container with id f423296e39accbd29a16e1e27d1d424a871110ba12c37f6d2b4f7c6c111da34e Jul 15 17:06:11.641776 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:11.641696 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" event={"ID":"cacf5e90-db63-4e20-a351-8f5aac61e289","Type":"ContainerStarted","Data":"f423296e39accbd29a16e1e27d1d424a871110ba12c37f6d2b4f7c6c111da34e"} Jul 15 17:06:14.655769 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:14.655739 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" event={"ID":"cacf5e90-db63-4e20-a351-8f5aac61e289","Type":"ContainerStarted","Data":"537882e50656be970663d01e8064b82b27ac965cccc8d894a554f310499eb597"} Jul 15 17:06:14.656137 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:14.655814 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" Jul 15 17:06:14.680376 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:14.680333 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" podStartSLOduration=1.674417754 podStartE2EDuration="4.680318139s" totalImagesPullingTime="3.005900385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:14.679050921 +0000 UTC m=+590.728048112" watchObservedRunningTime="2026-07-15 17:06:14.680318139 +0000 UTC m=+590.729315330" Jul 15 17:06:20.747586 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:20.747550 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64"] Jul 15 17:06:20.750946 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:20.750927 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:20.757242 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:20.757220 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 15 17:06:20.757493 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:20.757476 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 15 17:06:20.757565 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:20.757484 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 15 17:06:20.757740 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:20.757725 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8dsqb\"" Jul 15 17:06:20.762389 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:20.762374 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 17:06:20.775849 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:20.775821 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64"] Jul 15 17:06:20.919602 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:20.919567 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:20.919776 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:20.919642 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jcwf\" (UniqueName: \"kubernetes.io/projected/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-kube-api-access-8jcwf\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:20.919821 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:20.919771 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:20.919867 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:20.919831 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:20.919913 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:20.919897 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:20.919954 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:20.919930 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:20.919986 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:20.919959 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:21.020502 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.020403 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:21.020502 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.020442 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:21.020502 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.020465 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:21.020773 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.020593 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:21.020773 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.020628 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jcwf\" (UniqueName: \"kubernetes.io/projected/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-kube-api-access-8jcwf\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:21.020773 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.020728 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:21.020922 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.020785 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:21.021390 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.021361 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:21.022923 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.022901 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:21.023032 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.022965 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:21.023032 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.022966 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:21.023032 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.023023 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:21.029012 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.028988 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:21.029283 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.029264 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jcwf\" (UniqueName: \"kubernetes.io/projected/35f6e8b7-5a41-4696-9e24-a76d2eb42cb3-kube-api-access-8jcwf\") pod \"istiod-openshift-gateway-94bb8fbfd-bcq64\" (UID: \"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:21.059748 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.059725 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:21.188644 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.188618 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64"] Jul 15 17:06:21.190790 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:06:21.190763 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35f6e8b7_5a41_4696_9e24_a76d2eb42cb3.slice/crio-47d215e454d90f03b034bf951225d370ad3b9c78640664026a241e0408861747 WatchSource:0}: Error finding container 47d215e454d90f03b034bf951225d370ad3b9c78640664026a241e0408861747: Status 404 returned error can't find the container with id 47d215e454d90f03b034bf951225d370ad3b9c78640664026a241e0408861747 Jul 15 17:06:21.686255 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:21.686218 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" event={"ID":"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3","Type":"ContainerStarted","Data":"47d215e454d90f03b034bf951225d370ad3b9c78640664026a241e0408861747"} Jul 15 17:06:23.769603 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:23.769563 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:06:23.769861 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:23.769640 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 15 17:06:24.395283 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:24.395253 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:06:24.396514 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:24.396492 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:06:24.404552 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:24.404531 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:06:24.405940 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:24.405922 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:06:24.707187 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:24.707150 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" event={"ID":"35f6e8b7-5a41-4696-9e24-a76d2eb42cb3","Type":"ContainerStarted","Data":"dcfd6850c56497e65d2ff456298b00bc2c93cb1e4c0e361932f7388cf1c80728"} Jul 15 17:06:24.707425 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:24.707326 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:24.709358 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:24.709331 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" Jul 15 17:06:24.785019 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:24.784964 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bcq64" podStartSLOduration=2.208468954 podStartE2EDuration="4.784944636s" totalImagesPullingTime="2.576475682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:24.784650299 +0000 UTC m=+600.833647501" watchObservedRunningTime="2026-07-15 17:06:24.784944636 +0000 UTC m=+600.833941829" Jul 15 17:06:25.661954 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:06:25.661923 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5c485d79f6-zhw88" Jul 15 17:09:00.723634 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:00.723597 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:09:00.726915 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:00.726898 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-f7r67" Jul 15 17:09:00.729505 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:00.729485 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fx4k8\"" Jul 15 17:09:00.729614 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:00.729486 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 17:09:00.730271 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:00.730250 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 17:09:00.730379 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:00.730281 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 15 17:09:00.737069 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:00.737046 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:09:00.773046 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:00.773010 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:09:00.810486 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:00.810439 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64trv\" (UniqueName: \"kubernetes.io/projected/61667b9c-dd9c-453b-8829-85616cd057e9-kube-api-access-64trv\") pod \"limitador-limitador-54d6d988d5-f7r67\" (UID: \"61667b9c-dd9c-453b-8829-85616cd057e9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f7r67" Jul 15 17:09:00.810653 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:00.810498 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/61667b9c-dd9c-453b-8829-85616cd057e9-config-file\") pod \"limitador-limitador-54d6d988d5-f7r67\" (UID: \"61667b9c-dd9c-453b-8829-85616cd057e9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f7r67" Jul 15 17:09:00.911640 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:00.911601 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/61667b9c-dd9c-453b-8829-85616cd057e9-config-file\") pod \"limitador-limitador-54d6d988d5-f7r67\" (UID: \"61667b9c-dd9c-453b-8829-85616cd057e9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f7r67" Jul 15 17:09:00.911819 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:00.911732 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64trv\" (UniqueName: \"kubernetes.io/projected/61667b9c-dd9c-453b-8829-85616cd057e9-kube-api-access-64trv\") pod \"limitador-limitador-54d6d988d5-f7r67\" (UID: \"61667b9c-dd9c-453b-8829-85616cd057e9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f7r67" Jul 15 17:09:00.912244 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:00.912219 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/61667b9c-dd9c-453b-8829-85616cd057e9-config-file\") pod \"limitador-limitador-54d6d988d5-f7r67\" (UID: \"61667b9c-dd9c-453b-8829-85616cd057e9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f7r67" Jul 15 17:09:00.920481 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:00.920459 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64trv\" (UniqueName: \"kubernetes.io/projected/61667b9c-dd9c-453b-8829-85616cd057e9-kube-api-access-64trv\") pod \"limitador-limitador-54d6d988d5-f7r67\" (UID: \"61667b9c-dd9c-453b-8829-85616cd057e9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f7r67" Jul 15 17:09:01.038133 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:01.038049 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-f7r67" Jul 15 17:09:01.166176 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:01.166151 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:09:01.168117 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:09:01.168088 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61667b9c_dd9c_453b_8829_85616cd057e9.slice/crio-3c817a3995df3bf6de1ff2b790fc32d85d606698c8f401fcc12f9bad42b54743 WatchSource:0}: Error finding container 3c817a3995df3bf6de1ff2b790fc32d85d606698c8f401fcc12f9bad42b54743: Status 404 returned error can't find the container with id 3c817a3995df3bf6de1ff2b790fc32d85d606698c8f401fcc12f9bad42b54743 Jul 15 17:09:01.170181 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:01.170165 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 17:09:01.359410 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:01.359330 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-f7r67" event={"ID":"61667b9c-dd9c-453b-8829-85616cd057e9","Type":"ContainerStarted","Data":"3c817a3995df3bf6de1ff2b790fc32d85d606698c8f401fcc12f9bad42b54743"} Jul 15 17:09:04.381557 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:04.381518 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-f7r67" event={"ID":"61667b9c-dd9c-453b-8829-85616cd057e9","Type":"ContainerStarted","Data":"2af45ca1ddd2ce7c2c5e96a840c9c1a28d928277f4de89736018f07e3a7293ab"} Jul 15 17:09:04.381932 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:04.381662 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-f7r67" Jul 15 17:09:04.398554 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:04.398502 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-f7r67" podStartSLOduration=1.9235029300000002 podStartE2EDuration="4.398485221s" totalImagesPullingTime="2.474982291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:09:04.397460603 +0000 UTC m=+760.446457795" watchObservedRunningTime="2026-07-15 17:09:04.398485221 +0000 UTC m=+760.447482435" Jul 15 17:09:15.386940 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:09:15.386906 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-f7r67" Jul 15 17:10:21.009082 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:10:21.009046 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:10:56.593469 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:10:56.593432 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:11:05.903215 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:05.903181 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-swddx"] Jul 15 17:11:05.906686 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:05.906667 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:05.910007 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:05.909988 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k5dwz\"" Jul 15 17:11:05.910214 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:05.910200 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 17:11:05.910273 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:05.910210 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 15 17:11:05.910273 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:05.910228 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 17:11:05.916558 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:05.916534 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-swddx"] Jul 15 17:11:05.968125 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:05.968095 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a79e1a1c-d481-4036-8e89-7f37e22733ff-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:05.968300 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:05.968140 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a79e1a1c-d481-4036-8e89-7f37e22733ff-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:05.968300 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:05.968163 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a79e1a1c-d481-4036-8e89-7f37e22733ff-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:05.968300 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:05.968184 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a79e1a1c-d481-4036-8e89-7f37e22733ff-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:05.968300 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:05.968238 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a79e1a1c-d481-4036-8e89-7f37e22733ff-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:05.968300 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:05.968262 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx955\" (UniqueName: \"kubernetes.io/projected/a79e1a1c-d481-4036-8e89-7f37e22733ff-kube-api-access-gx955\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:06.072509 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:06.072473 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a79e1a1c-d481-4036-8e89-7f37e22733ff-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:06.072675 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:06.072522 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a79e1a1c-d481-4036-8e89-7f37e22733ff-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:06.072675 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:06.072540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx955\" (UniqueName: \"kubernetes.io/projected/a79e1a1c-d481-4036-8e89-7f37e22733ff-kube-api-access-gx955\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:06.072675 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:06.072618 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a79e1a1c-d481-4036-8e89-7f37e22733ff-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:06.072675 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:06.072656 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a79e1a1c-d481-4036-8e89-7f37e22733ff-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:06.072870 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:06.072690 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a79e1a1c-d481-4036-8e89-7f37e22733ff-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:06.072958 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:06.072935 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a79e1a1c-d481-4036-8e89-7f37e22733ff-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:06.073029 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:06.073002 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a79e1a1c-d481-4036-8e89-7f37e22733ff-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:06.073029 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:06.073020 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a79e1a1c-d481-4036-8e89-7f37e22733ff-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:06.074873 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:06.074852 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a79e1a1c-d481-4036-8e89-7f37e22733ff-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:06.075126 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:06.075109 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a79e1a1c-d481-4036-8e89-7f37e22733ff-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:06.081521 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:06.081484 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx955\" (UniqueName: \"kubernetes.io/projected/a79e1a1c-d481-4036-8e89-7f37e22733ff-kube-api-access-gx955\") pod \"e2e-distinct-simulated-kserve-59975d69f9-swddx\" (UID: \"a79e1a1c-d481-4036-8e89-7f37e22733ff\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:06.216933 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:06.216898 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:06.344805 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:06.344783 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-swddx"] Jul 15 17:11:06.347020 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:11:06.346992 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda79e1a1c_d481_4036_8e89_7f37e22733ff.slice/crio-1a25bef1014ef45723f8f7b90a19a91cf49b54b8f80ce543743bbd16e5c28144 WatchSource:0}: Error finding container 1a25bef1014ef45723f8f7b90a19a91cf49b54b8f80ce543743bbd16e5c28144: Status 404 returned error can't find the container with id 1a25bef1014ef45723f8f7b90a19a91cf49b54b8f80ce543743bbd16e5c28144 Jul 15 17:11:06.893078 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:06.893046 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:11:06.902431 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:06.902395 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" event={"ID":"a79e1a1c-d481-4036-8e89-7f37e22733ff","Type":"ContainerStarted","Data":"1a25bef1014ef45723f8f7b90a19a91cf49b54b8f80ce543743bbd16e5c28144"} Jul 15 17:11:07.909027 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:07.908989 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" event={"ID":"a79e1a1c-d481-4036-8e89-7f37e22733ff","Type":"ContainerStarted","Data":"4644febbb82f31a2b6c1b51d1cc0f811e8c86f9effd5338ec996c06f1b32039a"} Jul 15 17:11:07.930232 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:07.930187 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" podStartSLOduration=1.872950389 podStartE2EDuration="2.930171955s" totalImagesPullingTime="1.057221566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:11:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:11:07.928494641 +0000 UTC m=+883.977491834" watchObservedRunningTime="2026-07-15 17:11:07.930171955 +0000 UTC m=+883.979169145" Jul 15 17:11:13.996278 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:13.996216 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:11:16.217509 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:16.217425 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:16.217877 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:16.217773 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:16.229803 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:16.229778 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:16.962684 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:16.962658 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-swddx" Jul 15 17:11:19.889220 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:19.889186 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:11:23.484332 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.484300 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2"] Jul 15 17:11:23.486798 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.486777 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.488932 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.488910 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 15 17:11:23.497244 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.497223 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2"] Jul 15 17:11:23.527215 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.527191 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f74917-92f1-4535-b5e7-86eb9389555a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.527384 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.527247 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmm4t\" (UniqueName: \"kubernetes.io/projected/09f74917-92f1-4535-b5e7-86eb9389555a-kube-api-access-gmm4t\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.527384 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.527327 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f74917-92f1-4535-b5e7-86eb9389555a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.527384 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.527370 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f74917-92f1-4535-b5e7-86eb9389555a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.527529 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.527399 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f74917-92f1-4535-b5e7-86eb9389555a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.527529 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.527428 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f74917-92f1-4535-b5e7-86eb9389555a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.628641 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.628603 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f74917-92f1-4535-b5e7-86eb9389555a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.628789 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.628676 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmm4t\" (UniqueName: \"kubernetes.io/projected/09f74917-92f1-4535-b5e7-86eb9389555a-kube-api-access-gmm4t\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.628789 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.628696 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f74917-92f1-4535-b5e7-86eb9389555a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.628789 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.628713 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f74917-92f1-4535-b5e7-86eb9389555a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.628789 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.628731 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f74917-92f1-4535-b5e7-86eb9389555a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.628983 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.628879 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f74917-92f1-4535-b5e7-86eb9389555a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.629158 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.629137 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f74917-92f1-4535-b5e7-86eb9389555a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.629219 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.629160 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f74917-92f1-4535-b5e7-86eb9389555a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.629254 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.629226 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f74917-92f1-4535-b5e7-86eb9389555a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.630881 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.630857 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f74917-92f1-4535-b5e7-86eb9389555a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.631035 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.631018 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f74917-92f1-4535-b5e7-86eb9389555a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.637039 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.637018 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmm4t\" (UniqueName: \"kubernetes.io/projected/09f74917-92f1-4535-b5e7-86eb9389555a-kube-api-access-gmm4t\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-z7qv2\" (UID: \"09f74917-92f1-4535-b5e7-86eb9389555a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.797945 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.797857 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:23.925880 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.925855 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2"] Jul 15 17:11:23.927911 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:11:23.927876 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09f74917_92f1_4535_b5e7_86eb9389555a.slice/crio-04e987b323bec8d16a5a95d8f981bfc6913d97ae8d7938b13de72f1b758caf6f WatchSource:0}: Error finding container 04e987b323bec8d16a5a95d8f981bfc6913d97ae8d7938b13de72f1b758caf6f: Status 404 returned error can't find the container with id 04e987b323bec8d16a5a95d8f981bfc6913d97ae8d7938b13de72f1b758caf6f Jul 15 17:11:23.982665 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:23.982636 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" event={"ID":"09f74917-92f1-4535-b5e7-86eb9389555a","Type":"ContainerStarted","Data":"04e987b323bec8d16a5a95d8f981bfc6913d97ae8d7938b13de72f1b758caf6f"} Jul 15 17:11:24.435995 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:24.435970 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:11:24.435995 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:24.435994 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:11:24.444908 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:24.444892 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:11:24.445279 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:24.445265 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:11:25.401398 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:25.401362 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:11:26.997156 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:26.997115 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" event={"ID":"09f74917-92f1-4535-b5e7-86eb9389555a","Type":"ContainerStarted","Data":"aa24925c3e874c538ff71fbdc6997ea9f40d66c3bf4729b3d4e4b562b0b78a21"} Jul 15 17:11:27.016815 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:27.016769 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" podStartSLOduration=1.173836061 podStartE2EDuration="4.01675328s" totalImagesPullingTime="2.842917219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:11:27.015676942 +0000 UTC m=+903.064674133" watchObservedRunningTime="2026-07-15 17:11:27.01675328 +0000 UTC m=+903.065750470" Jul 15 17:11:33.798889 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:33.798851 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:33.799370 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:33.798909 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:33.811168 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:33.811145 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:34.037919 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:34.037890 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-z7qv2" Jul 15 17:11:42.395481 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:11:42.395448 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:13:03.208582 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:13:03.208547 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:13:15.297781 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:13:15.297745 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:13:24.991509 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:13:24.991472 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:13:33.786937 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:13:33.786904 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:13:44.792566 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:13:44.792532 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:14:12.328871 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:14:12.328836 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:14:26.887889 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:14:26.887806 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:15:05.197078 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:15:05.197040 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:15:14.492074 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:15:14.492042 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:15:26.793477 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:15:26.793442 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:15:38.698948 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:15:38.698917 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:16:06.192763 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:16:06.192729 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:16:15.997286 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:16:15.997252 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:16:24.463586 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:16:24.463557 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:16:24.465628 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:16:24.465606 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:16:24.478245 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:16:24.478229 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:16:24.480020 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:16:24.480002 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:16:38.294825 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:16:38.294788 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:16:49.923410 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:16:49.923375 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:17:14.193676 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:17:14.193635 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:17:25.891219 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:17:25.891111 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:17:41.520919 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:17:41.520878 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:17:50.996307 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:17:50.996267 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:18:01.187454 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:18:01.187413 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:18:10.482948 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:18:10.482913 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:18:19.603174 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:18:19.603138 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:20:06.891758 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:20:06.891721 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:20:18.202160 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:20:18.202078 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:20:27.097601 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:20:27.097565 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:20:41.095045 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:20:41.095012 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:20:50.293557 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:20:50.293522 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:21:01.220630 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:21:01.220595 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:21:09.696586 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:21:09.696558 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:21:20.617059 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:21:20.617026 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:21:24.499451 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:21:24.499421 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:21:24.502553 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:21:24.502530 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:21:24.508275 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:21:24.508256 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:21:24.510774 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:21:24.510756 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:21:32.996663 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:21:32.996632 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:21:41.696130 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:21:41.696099 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:21:52.291710 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:21:52.291674 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:22:16.397393 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:22:16.397358 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:22:23.691039 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:22:23.691003 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:22:32.798455 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:22:32.798423 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:22:43.501751 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:22:43.497753 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:23:01.095836 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:23:01.095799 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:23:12.092981 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:23:12.092948 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:23:21.296031 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:23:21.295946 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:23:38.298698 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:23:38.298662 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:23:45.717059 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:23:45.717027 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:23:55.601324 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:23:55.601279 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:24:08.700721 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:24:08.700687 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:24:18.297313 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:24:18.297263 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:24:26.890644 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:24:26.890610 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:24:39.394895 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:24:39.394862 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:24:50.993619 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:24:50.993538 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:25:03.898986 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:25:03.898946 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:25:15.992268 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:25:15.992230 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:25:27.897228 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:25:27.897194 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:25:39.690670 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:25:39.690632 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:25:52.290599 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:25:52.290565 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:26:08.499374 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:26:08.499338 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:26:21.295016 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:26:21.294980 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:26:24.532413 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:26:24.532382 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:26:24.533993 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:26:24.533971 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:26:24.541398 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:26:24.541345 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:26:24.543347 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:26:24.543326 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:27:22.295345 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:22.295310 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:27:40.745813 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:40.745766 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk"] Jul 15 17:27:40.749345 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:40.749322 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk" Jul 15 17:27:40.751981 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:40.751959 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:27:40.752123 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:40.752088 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:27:40.758624 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:40.758604 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk"] Jul 15 17:27:40.837235 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:40.837203 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82a12c9d-5cb9-4dac-aed2-57980169c6e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk\" (UID: \"82a12c9d-5cb9-4dac-aed2-57980169c6e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk" Jul 15 17:27:40.938516 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:40.938436 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82a12c9d-5cb9-4dac-aed2-57980169c6e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk\" (UID: \"82a12c9d-5cb9-4dac-aed2-57980169c6e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk" Jul 15 17:27:40.939050 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:40.939028 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82a12c9d-5cb9-4dac-aed2-57980169c6e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk\" (UID: \"82a12c9d-5cb9-4dac-aed2-57980169c6e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk" Jul 15 17:27:41.060533 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:41.060500 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk" Jul 15 17:27:41.186384 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:41.186351 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk"] Jul 15 17:27:41.189473 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:27:41.189404 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82a12c9d_5cb9_4dac_aed2_57980169c6e7.slice/crio-126c86218e30f3297956d0578ae2928d55b61c2b3e6441265ca144cca99f67f5 WatchSource:0}: Error finding container 126c86218e30f3297956d0578ae2928d55b61c2b3e6441265ca144cca99f67f5: Status 404 returned error can't find the container with id 126c86218e30f3297956d0578ae2928d55b61c2b3e6441265ca144cca99f67f5 Jul 15 17:27:41.191225 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:41.191207 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 17:27:41.237177 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:41.237149 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk" event={"ID":"82a12c9d-5cb9-4dac-aed2-57980169c6e7","Type":"ContainerStarted","Data":"126c86218e30f3297956d0578ae2928d55b61c2b3e6441265ca144cca99f67f5"} Jul 15 17:27:43.394606 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:43.394548 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:27:44.252026 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:44.251991 2562 generic.go:356] "Generic (PLEG): container finished" podID="82a12c9d-5cb9-4dac-aed2-57980169c6e7" containerID="f5ccd6d0c55aea4981a76d29ac9d219026b30d866bc7ec264efcc2e1ed34faa0" exitCode=7 Jul 15 17:27:44.252201 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:44.252076 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk" event={"ID":"82a12c9d-5cb9-4dac-aed2-57980169c6e7","Type":"ContainerDied","Data":"f5ccd6d0c55aea4981a76d29ac9d219026b30d866bc7ec264efcc2e1ed34faa0"} Jul 15 17:27:44.252262 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:44.252248 2562 scope.go:117] "RemoveContainer" containerID="f5ccd6d0c55aea4981a76d29ac9d219026b30d866bc7ec264efcc2e1ed34faa0" Jul 15 17:27:45.263238 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:45.263139 2562 generic.go:356] "Generic (PLEG): container finished" podID="82a12c9d-5cb9-4dac-aed2-57980169c6e7" containerID="0f6262ba2c130a8963366fafdafd3a0438ccfde579486489739f1854cea9e5b1" exitCode=0 Jul 15 17:27:45.263679 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:45.263226 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk" event={"ID":"82a12c9d-5cb9-4dac-aed2-57980169c6e7","Type":"ContainerDied","Data":"0f6262ba2c130a8963366fafdafd3a0438ccfde579486489739f1854cea9e5b1"} Jul 15 17:27:45.263679 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:45.263336 2562 scope.go:117] "RemoveContainer" containerID="f5ccd6d0c55aea4981a76d29ac9d219026b30d866bc7ec264efcc2e1ed34faa0" Jul 15 17:27:46.393430 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:46.393404 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk" Jul 15 17:27:46.488541 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:46.488514 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82a12c9d-5cb9-4dac-aed2-57980169c6e7-maas-api-service-ca\") pod \"82a12c9d-5cb9-4dac-aed2-57980169c6e7\" (UID: \"82a12c9d-5cb9-4dac-aed2-57980169c6e7\") " Jul 15 17:27:46.488840 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:46.488819 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82a12c9d-5cb9-4dac-aed2-57980169c6e7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "82a12c9d-5cb9-4dac-aed2-57980169c6e7" (UID: "82a12c9d-5cb9-4dac-aed2-57980169c6e7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:27:46.590057 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:46.590035 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82a12c9d-5cb9-4dac-aed2-57980169c6e7-maas-api-service-ca\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:27:47.274139 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:47.274113 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk" Jul 15 17:27:47.274139 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:47.274121 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk" event={"ID":"82a12c9d-5cb9-4dac-aed2-57980169c6e7","Type":"ContainerDied","Data":"126c86218e30f3297956d0578ae2928d55b61c2b3e6441265ca144cca99f67f5"} Jul 15 17:27:47.274358 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:47.274165 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="126c86218e30f3297956d0578ae2928d55b61c2b3e6441265ca144cca99f67f5" Jul 15 17:27:53.692871 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:27:53.692835 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:28:12.554426 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:12.554394 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl"] Jul 15 17:28:12.554931 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:12.554740 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82a12c9d-5cb9-4dac-aed2-57980169c6e7" containerName="revoke-keys" Jul 15 17:28:12.554931 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:12.554755 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="82a12c9d-5cb9-4dac-aed2-57980169c6e7" containerName="revoke-keys" Jul 15 17:28:12.554931 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:12.554769 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82a12c9d-5cb9-4dac-aed2-57980169c6e7" containerName="revoke-keys" Jul 15 17:28:12.554931 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:12.554774 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="82a12c9d-5cb9-4dac-aed2-57980169c6e7" containerName="revoke-keys" Jul 15 17:28:12.554931 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:12.554836 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="82a12c9d-5cb9-4dac-aed2-57980169c6e7" containerName="revoke-keys" Jul 15 17:28:12.554931 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:12.554844 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="82a12c9d-5cb9-4dac-aed2-57980169c6e7" containerName="revoke-keys" Jul 15 17:28:12.558383 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:12.558360 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl" Jul 15 17:28:12.560656 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:12.560636 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:28:12.560773 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:12.560635 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:28:12.568661 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:12.568637 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl"] Jul 15 17:28:12.717837 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:12.717769 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c51051e7-b80b-4e0c-8927-5843f8a45ed4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e7083808-ds4nl\" (UID: \"c51051e7-b80b-4e0c-8927-5843f8a45ed4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl" Jul 15 17:28:12.818646 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:12.818615 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c51051e7-b80b-4e0c-8927-5843f8a45ed4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e7083808-ds4nl\" (UID: \"c51051e7-b80b-4e0c-8927-5843f8a45ed4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl" Jul 15 17:28:12.819278 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:12.819259 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c51051e7-b80b-4e0c-8927-5843f8a45ed4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e7083808-ds4nl\" (UID: \"c51051e7-b80b-4e0c-8927-5843f8a45ed4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl" Jul 15 17:28:12.870829 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:12.870805 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl" Jul 15 17:28:12.993281 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:12.993199 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl"] Jul 15 17:28:12.996260 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:28:12.996234 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc51051e7_b80b_4e0c_8927_5843f8a45ed4.slice/crio-d671db9f718133d1fc477fc7abe67893d9060d9d08ad9a080ce94675c2319b84 WatchSource:0}: Error finding container d671db9f718133d1fc477fc7abe67893d9060d9d08ad9a080ce94675c2319b84: Status 404 returned error can't find the container with id d671db9f718133d1fc477fc7abe67893d9060d9d08ad9a080ce94675c2319b84 Jul 15 17:28:13.394365 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:13.394323 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl" event={"ID":"c51051e7-b80b-4e0c-8927-5843f8a45ed4","Type":"ContainerStarted","Data":"082a7fa4b2039deda59b0e489006fc7d9b58030701c5864d00b566f45f1ca824"} Jul 15 17:28:13.394530 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:13.394426 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl" event={"ID":"c51051e7-b80b-4e0c-8927-5843f8a45ed4","Type":"ContainerStarted","Data":"d671db9f718133d1fc477fc7abe67893d9060d9d08ad9a080ce94675c2319b84"} Jul 15 17:28:13.417371 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:13.417318 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl" podStartSLOduration=1.417279249 podStartE2EDuration="1.417279249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:28:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:28:13.4160736 +0000 UTC m=+1909.465070791" watchObservedRunningTime="2026-07-15 17:28:13.417279249 +0000 UTC m=+1909.466276441" Jul 15 17:28:14.399831 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:14.399795 2562 generic.go:356] "Generic (PLEG): container finished" podID="c51051e7-b80b-4e0c-8927-5843f8a45ed4" containerID="082a7fa4b2039deda59b0e489006fc7d9b58030701c5864d00b566f45f1ca824" exitCode=7 Jul 15 17:28:14.400195 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:14.399838 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl" event={"ID":"c51051e7-b80b-4e0c-8927-5843f8a45ed4","Type":"ContainerDied","Data":"082a7fa4b2039deda59b0e489006fc7d9b58030701c5864d00b566f45f1ca824"} Jul 15 17:28:14.400195 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:14.400025 2562 scope.go:117] "RemoveContainer" containerID="082a7fa4b2039deda59b0e489006fc7d9b58030701c5864d00b566f45f1ca824" Jul 15 17:28:14.994967 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:14.994938 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:28:15.406203 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:15.406171 2562 generic.go:356] "Generic (PLEG): container finished" podID="c51051e7-b80b-4e0c-8927-5843f8a45ed4" containerID="b2601a13cb90076173125fbbb9e16e35f694f87808074dd7e1fd739ae9519542" exitCode=7 Jul 15 17:28:15.406585 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:15.406255 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl" event={"ID":"c51051e7-b80b-4e0c-8927-5843f8a45ed4","Type":"ContainerDied","Data":"b2601a13cb90076173125fbbb9e16e35f694f87808074dd7e1fd739ae9519542"} Jul 15 17:28:15.406585 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:15.406334 2562 scope.go:117] "RemoveContainer" containerID="082a7fa4b2039deda59b0e489006fc7d9b58030701c5864d00b566f45f1ca824" Jul 15 17:28:15.406585 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:15.406503 2562 scope.go:117] "RemoveContainer" containerID="b2601a13cb90076173125fbbb9e16e35f694f87808074dd7e1fd739ae9519542" Jul 15 17:28:15.406710 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:28:15.406692 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e7083808-ds4nl_odh-ai-gateway-infra(c51051e7-b80b-4e0c-8927-5843f8a45ed4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl" podUID="c51051e7-b80b-4e0c-8927-5843f8a45ed4" Jul 15 17:28:16.412447 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:16.412421 2562 scope.go:117] "RemoveContainer" containerID="b2601a13cb90076173125fbbb9e16e35f694f87808074dd7e1fd739ae9519542" Jul 15 17:28:16.412798 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:28:16.412601 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e7083808-ds4nl_odh-ai-gateway-infra(c51051e7-b80b-4e0c-8927-5843f8a45ed4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl" podUID="c51051e7-b80b-4e0c-8927-5843f8a45ed4" Jul 15 17:28:28.561525 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:28.561497 2562 scope.go:117] "RemoveContainer" containerID="b2601a13cb90076173125fbbb9e16e35f694f87808074dd7e1fd739ae9519542" Jul 15 17:28:29.477676 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:29.477606 2562 generic.go:356] "Generic (PLEG): container finished" podID="c51051e7-b80b-4e0c-8927-5843f8a45ed4" containerID="976e6776650ef61d69d6da30222649dec0d03ea1dcf5658705f6d8ca937b4fbb" exitCode=0 Jul 15 17:28:29.477676 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:29.477646 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl" event={"ID":"c51051e7-b80b-4e0c-8927-5843f8a45ed4","Type":"ContainerDied","Data":"976e6776650ef61d69d6da30222649dec0d03ea1dcf5658705f6d8ca937b4fbb"} Jul 15 17:28:29.477838 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:29.477693 2562 scope.go:117] "RemoveContainer" containerID="b2601a13cb90076173125fbbb9e16e35f694f87808074dd7e1fd739ae9519542" Jul 15 17:28:30.501579 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:30.501550 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl"] Jul 15 17:28:30.620032 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:30.620009 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl" Jul 15 17:28:30.674168 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:30.674135 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c51051e7-b80b-4e0c-8927-5843f8a45ed4-maas-api-service-ca\") pod \"c51051e7-b80b-4e0c-8927-5843f8a45ed4\" (UID: \"c51051e7-b80b-4e0c-8927-5843f8a45ed4\") " Jul 15 17:28:30.674514 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:30.674491 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c51051e7-b80b-4e0c-8927-5843f8a45ed4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c51051e7-b80b-4e0c-8927-5843f8a45ed4" (UID: "c51051e7-b80b-4e0c-8927-5843f8a45ed4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:28:30.775697 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:30.775634 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c51051e7-b80b-4e0c-8927-5843f8a45ed4-maas-api-service-ca\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:28:31.489660 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:31.489629 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl" Jul 15 17:28:31.489848 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:31.489679 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl" event={"ID":"c51051e7-b80b-4e0c-8927-5843f8a45ed4","Type":"ContainerDied","Data":"d671db9f718133d1fc477fc7abe67893d9060d9d08ad9a080ce94675c2319b84"} Jul 15 17:28:31.489848 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:31.489748 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d671db9f718133d1fc477fc7abe67893d9060d9d08ad9a080ce94675c2319b84" Jul 15 17:28:31.522626 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:31.522600 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl"] Jul 15 17:28:31.525640 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:31.525619 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7083808-ds4nl"] Jul 15 17:28:32.566545 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:28:32.566508 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c51051e7-b80b-4e0c-8927-5843f8a45ed4" path="/var/lib/kubelet/pods/c51051e7-b80b-4e0c-8927-5843f8a45ed4/volumes" Jul 15 17:29:16.396878 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:16.396792 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:29:34.558271 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.558231 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97"] Jul 15 17:29:34.558787 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.558751 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c51051e7-b80b-4e0c-8927-5843f8a45ed4" containerName="revoke-keys" Jul 15 17:29:34.558787 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.558772 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c51051e7-b80b-4e0c-8927-5843f8a45ed4" containerName="revoke-keys" Jul 15 17:29:34.558787 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.558786 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c51051e7-b80b-4e0c-8927-5843f8a45ed4" containerName="revoke-keys" Jul 15 17:29:34.558943 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.558794 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c51051e7-b80b-4e0c-8927-5843f8a45ed4" containerName="revoke-keys" Jul 15 17:29:34.558943 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.558893 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c51051e7-b80b-4e0c-8927-5843f8a45ed4" containerName="revoke-keys" Jul 15 17:29:34.558943 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.558905 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c51051e7-b80b-4e0c-8927-5843f8a45ed4" containerName="revoke-keys" Jul 15 17:29:34.558943 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.558916 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c51051e7-b80b-4e0c-8927-5843f8a45ed4" containerName="revoke-keys" Jul 15 17:29:34.562421 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.562401 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97" Jul 15 17:29:34.564897 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.564876 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:29:34.565001 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.564926 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:29:34.569455 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.569432 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97"] Jul 15 17:29:34.702011 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.701980 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2984df39-0f53-4dca-8df7-23ca9a527ce4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-65f681ff-whv97\" (UID: \"2984df39-0f53-4dca-8df7-23ca9a527ce4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97" Jul 15 17:29:34.803284 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.803251 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2984df39-0f53-4dca-8df7-23ca9a527ce4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-65f681ff-whv97\" (UID: \"2984df39-0f53-4dca-8df7-23ca9a527ce4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97" Jul 15 17:29:34.803831 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.803811 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2984df39-0f53-4dca-8df7-23ca9a527ce4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-65f681ff-whv97\" (UID: \"2984df39-0f53-4dca-8df7-23ca9a527ce4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97" Jul 15 17:29:34.873839 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.873773 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97" Jul 15 17:29:34.999686 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:34.999662 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97"] Jul 15 17:29:35.001912 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:29:35.001882 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2984df39_0f53_4dca_8df7_23ca9a527ce4.slice/crio-637dd7ad83c5d41d9c1268c7b6825c4c4dfff904a44905587f44ed4d7c6edad2 WatchSource:0}: Error finding container 637dd7ad83c5d41d9c1268c7b6825c4c4dfff904a44905587f44ed4d7c6edad2: Status 404 returned error can't find the container with id 637dd7ad83c5d41d9c1268c7b6825c4c4dfff904a44905587f44ed4d7c6edad2 Jul 15 17:29:35.765070 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:35.765040 2562 generic.go:356] "Generic (PLEG): container finished" podID="2984df39-0f53-4dca-8df7-23ca9a527ce4" containerID="066a950cf47d9c6a1a15d96a0a703ed5af6df86a3889439d307516bfbe7f8f07" exitCode=7 Jul 15 17:29:35.765427 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:35.765096 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97" event={"ID":"2984df39-0f53-4dca-8df7-23ca9a527ce4","Type":"ContainerDied","Data":"066a950cf47d9c6a1a15d96a0a703ed5af6df86a3889439d307516bfbe7f8f07"} Jul 15 17:29:35.765427 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:35.765148 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97" event={"ID":"2984df39-0f53-4dca-8df7-23ca9a527ce4","Type":"ContainerStarted","Data":"637dd7ad83c5d41d9c1268c7b6825c4c4dfff904a44905587f44ed4d7c6edad2"} Jul 15 17:29:35.765427 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:35.765266 2562 scope.go:117] "RemoveContainer" containerID="066a950cf47d9c6a1a15d96a0a703ed5af6df86a3889439d307516bfbe7f8f07" Jul 15 17:29:36.771446 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:36.771417 2562 generic.go:356] "Generic (PLEG): container finished" podID="2984df39-0f53-4dca-8df7-23ca9a527ce4" containerID="3d3a39f2275af651b5dddb08fe281428f951f79ff1bae76c1221b6ab478b321c" exitCode=7 Jul 15 17:29:36.771841 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:36.771483 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97" event={"ID":"2984df39-0f53-4dca-8df7-23ca9a527ce4","Type":"ContainerDied","Data":"3d3a39f2275af651b5dddb08fe281428f951f79ff1bae76c1221b6ab478b321c"} Jul 15 17:29:36.771841 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:36.771523 2562 scope.go:117] "RemoveContainer" containerID="066a950cf47d9c6a1a15d96a0a703ed5af6df86a3889439d307516bfbe7f8f07" Jul 15 17:29:36.771841 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:36.771762 2562 scope.go:117] "RemoveContainer" containerID="3d3a39f2275af651b5dddb08fe281428f951f79ff1bae76c1221b6ab478b321c" Jul 15 17:29:36.771993 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:29:36.771952 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-65f681ff-whv97_odh-ai-gateway-infra(2984df39-0f53-4dca-8df7-23ca9a527ce4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97" podUID="2984df39-0f53-4dca-8df7-23ca9a527ce4" Jul 15 17:29:37.777985 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:37.777960 2562 scope.go:117] "RemoveContainer" containerID="3d3a39f2275af651b5dddb08fe281428f951f79ff1bae76c1221b6ab478b321c" Jul 15 17:29:37.778358 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:29:37.778139 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-65f681ff-whv97_odh-ai-gateway-infra(2984df39-0f53-4dca-8df7-23ca9a527ce4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97" podUID="2984df39-0f53-4dca-8df7-23ca9a527ce4" Jul 15 17:29:37.986532 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:37.986491 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:29:52.561568 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:52.561534 2562 scope.go:117] "RemoveContainer" containerID="3d3a39f2275af651b5dddb08fe281428f951f79ff1bae76c1221b6ab478b321c" Jul 15 17:29:52.843585 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:52.843528 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97" event={"ID":"2984df39-0f53-4dca-8df7-23ca9a527ce4","Type":"ContainerStarted","Data":"e67b549e313c6a0a5460e271cf0c53c8643b325e6152f4a114242489534da091"} Jul 15 17:29:52.860125 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:52.860085 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97" podStartSLOduration=18.860070362 podStartE2EDuration="18.860070362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:29:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:29:52.858437065 +0000 UTC m=+2008.907434257" watchObservedRunningTime="2026-07-15 17:29:52.860070362 +0000 UTC m=+2008.909067551" Jul 15 17:29:53.850131 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:53.850094 2562 generic.go:356] "Generic (PLEG): container finished" podID="2984df39-0f53-4dca-8df7-23ca9a527ce4" containerID="e67b549e313c6a0a5460e271cf0c53c8643b325e6152f4a114242489534da091" exitCode=0 Jul 15 17:29:53.850666 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:53.850161 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97" event={"ID":"2984df39-0f53-4dca-8df7-23ca9a527ce4","Type":"ContainerDied","Data":"e67b549e313c6a0a5460e271cf0c53c8643b325e6152f4a114242489534da091"} Jul 15 17:29:53.850666 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:53.850236 2562 scope.go:117] "RemoveContainer" containerID="3d3a39f2275af651b5dddb08fe281428f951f79ff1bae76c1221b6ab478b321c" Jul 15 17:29:53.875241 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:53.875219 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97"] Jul 15 17:29:54.979155 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:54.979131 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97" Jul 15 17:29:55.172071 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:55.172049 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2984df39-0f53-4dca-8df7-23ca9a527ce4-maas-api-service-ca\") pod \"2984df39-0f53-4dca-8df7-23ca9a527ce4\" (UID: \"2984df39-0f53-4dca-8df7-23ca9a527ce4\") " Jul 15 17:29:55.172400 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:55.172378 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2984df39-0f53-4dca-8df7-23ca9a527ce4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2984df39-0f53-4dca-8df7-23ca9a527ce4" (UID: "2984df39-0f53-4dca-8df7-23ca9a527ce4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:29:55.272952 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:55.272927 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2984df39-0f53-4dca-8df7-23ca9a527ce4-maas-api-service-ca\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:29:55.861907 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:55.861876 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97" Jul 15 17:29:55.861907 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:55.861887 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97" event={"ID":"2984df39-0f53-4dca-8df7-23ca9a527ce4","Type":"ContainerDied","Data":"637dd7ad83c5d41d9c1268c7b6825c4c4dfff904a44905587f44ed4d7c6edad2"} Jul 15 17:29:55.862107 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:55.861930 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="637dd7ad83c5d41d9c1268c7b6825c4c4dfff904a44905587f44ed4d7c6edad2" Jul 15 17:29:55.891561 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:55.891537 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97"] Jul 15 17:29:55.896263 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:55.896240 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-whv97"] Jul 15 17:29:56.566925 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:29:56.566884 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2984df39-0f53-4dca-8df7-23ca9a527ce4" path="/var/lib/kubelet/pods/2984df39-0f53-4dca-8df7-23ca9a527ce4/volumes" Jul 15 17:30:00.139282 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.139252 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735610-vm6fk"] Jul 15 17:30:00.139641 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.139624 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2984df39-0f53-4dca-8df7-23ca9a527ce4" containerName="revoke-keys" Jul 15 17:30:00.139641 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.139636 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2984df39-0f53-4dca-8df7-23ca9a527ce4" containerName="revoke-keys" Jul 15 17:30:00.139743 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.139654 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2984df39-0f53-4dca-8df7-23ca9a527ce4" containerName="revoke-keys" Jul 15 17:30:00.139743 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.139659 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2984df39-0f53-4dca-8df7-23ca9a527ce4" containerName="revoke-keys" Jul 15 17:30:00.139743 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.139673 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c51051e7-b80b-4e0c-8927-5843f8a45ed4" containerName="revoke-keys" Jul 15 17:30:00.139743 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.139678 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c51051e7-b80b-4e0c-8927-5843f8a45ed4" containerName="revoke-keys" Jul 15 17:30:00.139743 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.139728 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2984df39-0f53-4dca-8df7-23ca9a527ce4" containerName="revoke-keys" Jul 15 17:30:00.139743 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.139735 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2984df39-0f53-4dca-8df7-23ca9a527ce4" containerName="revoke-keys" Jul 15 17:30:00.139743 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.139740 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2984df39-0f53-4dca-8df7-23ca9a527ce4" containerName="revoke-keys" Jul 15 17:30:00.142591 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.142572 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735610-vm6fk" Jul 15 17:30:00.154320 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.154281 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:30:00.161004 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.160985 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735610-vm6fk"] Jul 15 17:30:00.161777 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.161755 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735610-vm6fk" Jul 15 17:30:00.177726 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.177693 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-65f681ff-29735610-ggl88"] Jul 15 17:30:00.178216 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.178200 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2984df39-0f53-4dca-8df7-23ca9a527ce4" containerName="revoke-keys" Jul 15 17:30:00.178326 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.178220 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2984df39-0f53-4dca-8df7-23ca9a527ce4" containerName="revoke-keys" Jul 15 17:30:00.181566 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.181543 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-65f681ff-29735610-ggl88" Jul 15 17:30:00.190686 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.190665 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-65f681ff-29735610-ggl88"] Jul 15 17:30:00.196361 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.196343 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-65f681ff-29735610-ggl88" Jul 15 17:30:00.307948 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.307921 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735610-vm6fk"] Jul 15 17:30:00.310527 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:30:00.310498 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbf5c92d_e0a4_4eda_9a0e_004c41e261b8.slice/crio-9a5e762dcbea84be900df26dc5557b4a36f00444e5ad8514efe4f87df98e4b55 WatchSource:0}: Error finding container 9a5e762dcbea84be900df26dc5557b4a36f00444e5ad8514efe4f87df98e4b55: Status 404 returned error can't find the container with id 9a5e762dcbea84be900df26dc5557b4a36f00444e5ad8514efe4f87df98e4b55 Jul 15 17:30:00.333282 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.333257 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-65f681ff-29735610-ggl88"] Jul 15 17:30:00.341392 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:30:00.341361 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf86e3cb_b397_4527_bd9d_d2497a4fff3c.slice/crio-36b1c88e0acd4d24ccaef219f7465d93b9e3f8ad78e4b9a5153ab1ef22ca1c44 WatchSource:0}: Error finding container 36b1c88e0acd4d24ccaef219f7465d93b9e3f8ad78e4b9a5153ab1ef22ca1c44: Status 404 returned error can't find the container with id 36b1c88e0acd4d24ccaef219f7465d93b9e3f8ad78e4b9a5153ab1ef22ca1c44 Jul 15 17:30:00.886565 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.886488 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-65f681ff-29735610-ggl88" event={"ID":"df86e3cb-b397-4527-bd9d-d2497a4fff3c","Type":"ContainerStarted","Data":"3b3ac4ecfd8416f8516851879f635f0888fc26be9168b8b37813a75efc786a6c"} Jul 15 17:30:00.886565 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.886552 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-65f681ff-29735610-ggl88" event={"ID":"df86e3cb-b397-4527-bd9d-d2497a4fff3c","Type":"ContainerStarted","Data":"36b1c88e0acd4d24ccaef219f7465d93b9e3f8ad78e4b9a5153ab1ef22ca1c44"} Jul 15 17:30:00.887965 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.887941 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735610-vm6fk" event={"ID":"cbf5c92d-e0a4-4eda-9a0e-004c41e261b8","Type":"ContainerStarted","Data":"6d072adb42ae53c69330137839328c50b1510255f831abefbfbaec80bb17c554"} Jul 15 17:30:00.888075 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.887978 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735610-vm6fk" event={"ID":"cbf5c92d-e0a4-4eda-9a0e-004c41e261b8","Type":"ContainerStarted","Data":"9a5e762dcbea84be900df26dc5557b4a36f00444e5ad8514efe4f87df98e4b55"} Jul 15 17:30:00.904659 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.904600 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-65f681ff-29735610-ggl88" podStartSLOduration=0.904580276 podStartE2EDuration="904.580276ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:30:00.90227825 +0000 UTC m=+2016.951275441" watchObservedRunningTime="2026-07-15 17:30:00.904580276 +0000 UTC m=+2016.953577487" Jul 15 17:30:00.917673 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:00.917632 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735610-vm6fk" podStartSLOduration=0.91761861 podStartE2EDuration="917.61861ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:30:00.91581144 +0000 UTC m=+2016.964808642" watchObservedRunningTime="2026-07-15 17:30:00.91761861 +0000 UTC m=+2016.966615815" Jul 15 17:30:01.895015 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:01.894981 2562 generic.go:356] "Generic (PLEG): container finished" podID="df86e3cb-b397-4527-bd9d-d2497a4fff3c" containerID="3b3ac4ecfd8416f8516851879f635f0888fc26be9168b8b37813a75efc786a6c" exitCode=0 Jul 15 17:30:01.895473 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:01.895056 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-65f681ff-29735610-ggl88" event={"ID":"df86e3cb-b397-4527-bd9d-d2497a4fff3c","Type":"ContainerDied","Data":"3b3ac4ecfd8416f8516851879f635f0888fc26be9168b8b37813a75efc786a6c"} Jul 15 17:30:01.896473 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:01.896447 2562 generic.go:356] "Generic (PLEG): container finished" podID="cbf5c92d-e0a4-4eda-9a0e-004c41e261b8" containerID="6d072adb42ae53c69330137839328c50b1510255f831abefbfbaec80bb17c554" exitCode=0 Jul 15 17:30:01.896557 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:01.896526 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735610-vm6fk" event={"ID":"cbf5c92d-e0a4-4eda-9a0e-004c41e261b8","Type":"ContainerDied","Data":"6d072adb42ae53c69330137839328c50b1510255f831abefbfbaec80bb17c554"} Jul 15 17:30:03.036948 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:03.036926 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-65f681ff-29735610-ggl88" Jul 15 17:30:03.039633 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:03.039615 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735610-vm6fk" Jul 15 17:30:03.906901 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:03.906875 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-65f681ff-29735610-ggl88" Jul 15 17:30:03.907078 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:03.906882 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-65f681ff-29735610-ggl88" event={"ID":"df86e3cb-b397-4527-bd9d-d2497a4fff3c","Type":"ContainerDied","Data":"36b1c88e0acd4d24ccaef219f7465d93b9e3f8ad78e4b9a5153ab1ef22ca1c44"} Jul 15 17:30:03.907078 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:03.906989 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36b1c88e0acd4d24ccaef219f7465d93b9e3f8ad78e4b9a5153ab1ef22ca1c44" Jul 15 17:30:03.908104 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:03.908087 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735610-vm6fk" Jul 15 17:30:03.908195 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:03.908103 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735610-vm6fk" event={"ID":"cbf5c92d-e0a4-4eda-9a0e-004c41e261b8","Type":"ContainerDied","Data":"9a5e762dcbea84be900df26dc5557b4a36f00444e5ad8514efe4f87df98e4b55"} Jul 15 17:30:03.908195 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:03.908145 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a5e762dcbea84be900df26dc5557b4a36f00444e5ad8514efe4f87df98e4b55" Jul 15 17:30:34.615625 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:34.615591 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2"] Jul 15 17:30:34.616041 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:34.615950 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df86e3cb-b397-4527-bd9d-d2497a4fff3c" containerName="cleanup" Jul 15 17:30:34.616041 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:34.615961 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="df86e3cb-b397-4527-bd9d-d2497a4fff3c" containerName="cleanup" Jul 15 17:30:34.616041 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:34.615972 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbf5c92d-e0a4-4eda-9a0e-004c41e261b8" containerName="cleanup" Jul 15 17:30:34.616041 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:34.615978 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbf5c92d-e0a4-4eda-9a0e-004c41e261b8" containerName="cleanup" Jul 15 17:30:34.616041 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:34.616034 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbf5c92d-e0a4-4eda-9a0e-004c41e261b8" containerName="cleanup" Jul 15 17:30:34.616194 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:34.616045 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="df86e3cb-b397-4527-bd9d-d2497a4fff3c" containerName="cleanup" Jul 15 17:30:34.619190 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:34.619173 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2" Jul 15 17:30:34.621635 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:34.621614 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:30:34.621732 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:34.621614 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:30:34.627180 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:34.627157 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2"] Jul 15 17:30:34.686227 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:34.686193 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db3848df-93ec-4fb6-a527-d0392839ba93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2\" (UID: \"db3848df-93ec-4fb6-a527-d0392839ba93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2" Jul 15 17:30:34.787371 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:34.787341 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db3848df-93ec-4fb6-a527-d0392839ba93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2\" (UID: \"db3848df-93ec-4fb6-a527-d0392839ba93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2" Jul 15 17:30:34.787895 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:34.787877 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db3848df-93ec-4fb6-a527-d0392839ba93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2\" (UID: \"db3848df-93ec-4fb6-a527-d0392839ba93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2" Jul 15 17:30:34.930093 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:34.930073 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2" Jul 15 17:30:35.056955 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:35.056934 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2"] Jul 15 17:30:35.059266 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:30:35.059239 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb3848df_93ec_4fb6_a527_d0392839ba93.slice/crio-273321cea6971835156f23c2499248cdc4758aabaa2ea4a60f1fa255874b870d WatchSource:0}: Error finding container 273321cea6971835156f23c2499248cdc4758aabaa2ea4a60f1fa255874b870d: Status 404 returned error can't find the container with id 273321cea6971835156f23c2499248cdc4758aabaa2ea4a60f1fa255874b870d Jul 15 17:30:36.053022 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:36.052945 2562 generic.go:356] "Generic (PLEG): container finished" podID="db3848df-93ec-4fb6-a527-d0392839ba93" containerID="4bb53664323b77fc8e7ab234ac814e6f68002ae3187e27f1f430543d2134f9f1" exitCode=0 Jul 15 17:30:36.053022 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:36.052986 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2" event={"ID":"db3848df-93ec-4fb6-a527-d0392839ba93","Type":"ContainerDied","Data":"4bb53664323b77fc8e7ab234ac814e6f68002ae3187e27f1f430543d2134f9f1"} Jul 15 17:30:36.053423 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:36.053028 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2" event={"ID":"db3848df-93ec-4fb6-a527-d0392839ba93","Type":"ContainerStarted","Data":"273321cea6971835156f23c2499248cdc4758aabaa2ea4a60f1fa255874b870d"} Jul 15 17:30:37.183081 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:37.183056 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2" Jul 15 17:30:37.308073 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:37.308008 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db3848df-93ec-4fb6-a527-d0392839ba93-maas-api-service-ca\") pod \"db3848df-93ec-4fb6-a527-d0392839ba93\" (UID: \"db3848df-93ec-4fb6-a527-d0392839ba93\") " Jul 15 17:30:37.308283 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:37.308263 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db3848df-93ec-4fb6-a527-d0392839ba93-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "db3848df-93ec-4fb6-a527-d0392839ba93" (UID: "db3848df-93ec-4fb6-a527-d0392839ba93"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:30:37.408576 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:37.408548 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db3848df-93ec-4fb6-a527-d0392839ba93-maas-api-service-ca\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:30:38.062590 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:38.062558 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2" Jul 15 17:30:38.062780 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:38.062599 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2" event={"ID":"db3848df-93ec-4fb6-a527-d0392839ba93","Type":"ContainerDied","Data":"273321cea6971835156f23c2499248cdc4758aabaa2ea4a60f1fa255874b870d"} Jul 15 17:30:38.062780 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:38.062644 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="273321cea6971835156f23c2499248cdc4758aabaa2ea4a60f1fa255874b870d" Jul 15 17:30:45.208162 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:45.208126 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-65f681ff-29735610-ggl88"] Jul 15 17:30:45.211174 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:45.211146 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-65f681ff-29735610-ggl88"] Jul 15 17:30:46.565935 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:46.565845 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df86e3cb-b397-4527-bd9d-d2497a4fff3c" path="/var/lib/kubelet/pods/df86e3cb-b397-4527-bd9d-d2497a4fff3c/volumes" Jul 15 17:30:48.090597 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:30:48.090561 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:31:24.565718 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:31:24.565639 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:31:24.570522 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:31:24.570500 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:31:24.576056 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:31:24.576035 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:31:24.579203 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:31:24.579187 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:32:26.694592 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:32:26.694546 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:32:34.635795 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:32:34.635763 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:32:43.405699 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:32:43.405664 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:32:53.208133 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:32:53.208100 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:33:10.394351 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:33:10.394307 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:33:50.783989 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:33:50.783908 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:34:13.888336 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:13.888302 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:34:32.485073 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:32.485039 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:34:38.460505 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:38.460470 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r"] Jul 15 17:34:38.460866 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:38.460822 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db3848df-93ec-4fb6-a527-d0392839ba93" containerName="revoke-keys" Jul 15 17:34:38.460866 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:38.460833 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="db3848df-93ec-4fb6-a527-d0392839ba93" containerName="revoke-keys" Jul 15 17:34:38.460937 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:38.460900 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="db3848df-93ec-4fb6-a527-d0392839ba93" containerName="revoke-keys" Jul 15 17:34:38.463901 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:38.463885 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r" Jul 15 17:34:38.466219 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:38.466196 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:34:38.466401 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:38.466197 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:34:38.470499 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:38.470475 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r"] Jul 15 17:34:38.559652 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:38.559628 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0572ad09-7829-49f1-b246-b623f504a2bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r\" (UID: \"0572ad09-7829-49f1-b246-b623f504a2bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r" Jul 15 17:34:38.660668 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:38.660637 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0572ad09-7829-49f1-b246-b623f504a2bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r\" (UID: \"0572ad09-7829-49f1-b246-b623f504a2bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r" Jul 15 17:34:38.661238 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:38.661214 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0572ad09-7829-49f1-b246-b623f504a2bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r\" (UID: \"0572ad09-7829-49f1-b246-b623f504a2bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r" Jul 15 17:34:38.775123 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:38.775055 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r" Jul 15 17:34:39.106234 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:39.106210 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r"] Jul 15 17:34:39.108727 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:34:39.108701 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0572ad09_7829_49f1_b246_b623f504a2bd.slice/crio-01d23063dd302678979df2b36a8c8497236428b5b577d87a0c6742944fbe663c WatchSource:0}: Error finding container 01d23063dd302678979df2b36a8c8497236428b5b577d87a0c6742944fbe663c: Status 404 returned error can't find the container with id 01d23063dd302678979df2b36a8c8497236428b5b577d87a0c6742944fbe663c Jul 15 17:34:39.177793 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:39.177759 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r" event={"ID":"0572ad09-7829-49f1-b246-b623f504a2bd","Type":"ContainerStarted","Data":"01d23063dd302678979df2b36a8c8497236428b5b577d87a0c6742944fbe663c"} Jul 15 17:34:40.183271 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:40.183238 2562 generic.go:356] "Generic (PLEG): container finished" podID="0572ad09-7829-49f1-b246-b623f504a2bd" containerID="7a6d4e7c9f30ec992e492f2775c5a328c6bec0aff6eef377e998cedab378c413" exitCode=0 Jul 15 17:34:40.183646 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:40.183323 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r" event={"ID":"0572ad09-7829-49f1-b246-b623f504a2bd","Type":"ContainerDied","Data":"7a6d4e7c9f30ec992e492f2775c5a328c6bec0aff6eef377e998cedab378c413"} Jul 15 17:34:41.312489 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:41.312464 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r" Jul 15 17:34:41.383268 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:41.383236 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0572ad09-7829-49f1-b246-b623f504a2bd-maas-api-service-ca\") pod \"0572ad09-7829-49f1-b246-b623f504a2bd\" (UID: \"0572ad09-7829-49f1-b246-b623f504a2bd\") " Jul 15 17:34:41.383564 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:41.383544 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0572ad09-7829-49f1-b246-b623f504a2bd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0572ad09-7829-49f1-b246-b623f504a2bd" (UID: "0572ad09-7829-49f1-b246-b623f504a2bd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:34:41.484630 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:41.484549 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0572ad09-7829-49f1-b246-b623f504a2bd-maas-api-service-ca\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:34:42.193967 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:42.193930 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r" event={"ID":"0572ad09-7829-49f1-b246-b623f504a2bd","Type":"ContainerDied","Data":"01d23063dd302678979df2b36a8c8497236428b5b577d87a0c6742944fbe663c"} Jul 15 17:34:42.193967 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:42.193959 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r" Jul 15 17:34:42.194214 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:42.193981 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01d23063dd302678979df2b36a8c8497236428b5b577d87a0c6742944fbe663c" Jul 15 17:34:52.787874 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:34:52.787840 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:35:24.598353 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:35:24.598259 2562 scope.go:117] "RemoveContainer" containerID="976e6776650ef61d69d6da30222649dec0d03ea1dcf5658705f6d8ca937b4fbb" Jul 15 17:35:50.004157 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:35:50.000652 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:36:04.195308 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:36:04.195269 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:36:24.601132 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:36:24.601105 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:36:24.604940 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:36:24.604919 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:36:24.609778 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:36:24.609760 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:36:24.613170 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:36:24.613153 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:36:24.615716 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:36:24.615699 2562 scope.go:117] "RemoveContainer" containerID="e67b549e313c6a0a5460e271cf0c53c8643b325e6152f4a114242489534da091" Jul 15 17:36:24.623151 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:36:24.623135 2562 scope.go:117] "RemoveContainer" containerID="3b3ac4ecfd8416f8516851879f635f0888fc26be9168b8b37813a75efc786a6c" Jul 15 17:36:52.602974 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:36:52.602936 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:37:04.798656 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:04.798624 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:37:14.615917 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:14.615884 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:37:25.633407 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:25.633367 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:37:36.134373 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:36.134344 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:37:53.068178 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:53.068141 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn"] Jul 15 17:37:53.068632 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:53.068614 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0572ad09-7829-49f1-b246-b623f504a2bd" containerName="revoke-keys" Jul 15 17:37:53.068678 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:53.068635 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0572ad09-7829-49f1-b246-b623f504a2bd" containerName="revoke-keys" Jul 15 17:37:53.068770 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:53.068758 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0572ad09-7829-49f1-b246-b623f504a2bd" containerName="revoke-keys" Jul 15 17:37:53.071911 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:53.071893 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn" Jul 15 17:37:53.077417 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:53.077395 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:37:53.077517 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:53.077395 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:37:53.108631 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:53.108608 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn"] Jul 15 17:37:53.153570 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:53.153540 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1c1e4d8-503f-4971-a3db-289f81090a1c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn\" (UID: \"e1c1e4d8-503f-4971-a3db-289f81090a1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn" Jul 15 17:37:53.254800 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:53.254762 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1c1e4d8-503f-4971-a3db-289f81090a1c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn\" (UID: \"e1c1e4d8-503f-4971-a3db-289f81090a1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn" Jul 15 17:37:53.255450 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:53.255429 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1c1e4d8-503f-4971-a3db-289f81090a1c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn\" (UID: \"e1c1e4d8-503f-4971-a3db-289f81090a1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn" Jul 15 17:37:53.381728 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:53.381651 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn" Jul 15 17:37:53.540276 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:53.540243 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn"] Jul 15 17:37:53.541878 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:37:53.541847 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c1e4d8_503f_4971_a3db_289f81090a1c.slice/crio-5da26b81ecbb6f2899d990a116544068df4ec79285e494c37009d72082b15825 WatchSource:0}: Error finding container 5da26b81ecbb6f2899d990a116544068df4ec79285e494c37009d72082b15825: Status 404 returned error can't find the container with id 5da26b81ecbb6f2899d990a116544068df4ec79285e494c37009d72082b15825 Jul 15 17:37:54.074072 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:54.073976 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn" event={"ID":"e1c1e4d8-503f-4971-a3db-289f81090a1c","Type":"ContainerStarted","Data":"06795a18fbba270398108e14c01025dd160ffbe1b2c0fa7b54fd919a3f224874"} Jul 15 17:37:54.074072 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:54.074065 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn" event={"ID":"e1c1e4d8-503f-4971-a3db-289f81090a1c","Type":"ContainerStarted","Data":"5da26b81ecbb6f2899d990a116544068df4ec79285e494c37009d72082b15825"} Jul 15 17:37:54.112596 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:54.112546 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn" podStartSLOduration=1.11253117 podStartE2EDuration="1.11253117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:37:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:37:54.110120026 +0000 UTC m=+2490.159117218" watchObservedRunningTime="2026-07-15 17:37:54.11253117 +0000 UTC m=+2490.161528360" Jul 15 17:37:54.536823 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:54.536784 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:37:55.079648 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:55.079617 2562 generic.go:356] "Generic (PLEG): container finished" podID="e1c1e4d8-503f-4971-a3db-289f81090a1c" containerID="06795a18fbba270398108e14c01025dd160ffbe1b2c0fa7b54fd919a3f224874" exitCode=0 Jul 15 17:37:55.080077 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:55.079705 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn" event={"ID":"e1c1e4d8-503f-4971-a3db-289f81090a1c","Type":"ContainerDied","Data":"06795a18fbba270398108e14c01025dd160ffbe1b2c0fa7b54fd919a3f224874"} Jul 15 17:37:56.211610 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:56.211587 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn" Jul 15 17:37:56.286011 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:56.285982 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1c1e4d8-503f-4971-a3db-289f81090a1c-maas-api-service-ca\") pod \"e1c1e4d8-503f-4971-a3db-289f81090a1c\" (UID: \"e1c1e4d8-503f-4971-a3db-289f81090a1c\") " Jul 15 17:37:56.286325 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:56.286281 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1c1e4d8-503f-4971-a3db-289f81090a1c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e1c1e4d8-503f-4971-a3db-289f81090a1c" (UID: "e1c1e4d8-503f-4971-a3db-289f81090a1c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:37:56.387762 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:56.387669 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1c1e4d8-503f-4971-a3db-289f81090a1c-maas-api-service-ca\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:37:57.090625 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:57.090598 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn" Jul 15 17:37:57.090799 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:57.090597 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn" event={"ID":"e1c1e4d8-503f-4971-a3db-289f81090a1c","Type":"ContainerDied","Data":"5da26b81ecbb6f2899d990a116544068df4ec79285e494c37009d72082b15825"} Jul 15 17:37:57.090799 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:37:57.090722 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5da26b81ecbb6f2899d990a116544068df4ec79285e494c37009d72082b15825" Jul 15 17:38:09.252947 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:09.252914 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:38:19.726880 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:19.726803 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck"] Jul 15 17:38:19.727218 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:19.727159 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1c1e4d8-503f-4971-a3db-289f81090a1c" containerName="revoke-keys" Jul 15 17:38:19.727218 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:19.727170 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1c1e4d8-503f-4971-a3db-289f81090a1c" containerName="revoke-keys" Jul 15 17:38:19.727302 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:19.727241 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1c1e4d8-503f-4971-a3db-289f81090a1c" containerName="revoke-keys" Jul 15 17:38:19.730355 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:19.730337 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck" Jul 15 17:38:19.741732 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:19.741706 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:38:19.742304 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:19.742274 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:38:19.778181 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:19.778158 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck"] Jul 15 17:38:19.789410 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:19.789387 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69981e25-a1fb-4e6d-b728-8b15386fa5f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck\" (UID: \"69981e25-a1fb-4e6d-b728-8b15386fa5f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck" Jul 15 17:38:19.890831 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:19.890797 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69981e25-a1fb-4e6d-b728-8b15386fa5f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck\" (UID: \"69981e25-a1fb-4e6d-b728-8b15386fa5f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck" Jul 15 17:38:19.891382 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:19.891366 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69981e25-a1fb-4e6d-b728-8b15386fa5f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck\" (UID: \"69981e25-a1fb-4e6d-b728-8b15386fa5f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck" Jul 15 17:38:20.045665 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:20.045594 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck" Jul 15 17:38:20.197932 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:20.197899 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck"] Jul 15 17:38:20.199945 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:38:20.199909 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69981e25_a1fb_4e6d_b728_8b15386fa5f6.slice/crio-df28e89816b7888cf3664f01a21d35c3bb3020fce395adba33a614c5cb0a13ef WatchSource:0}: Error finding container df28e89816b7888cf3664f01a21d35c3bb3020fce395adba33a614c5cb0a13ef: Status 404 returned error can't find the container with id df28e89816b7888cf3664f01a21d35c3bb3020fce395adba33a614c5cb0a13ef Jul 15 17:38:21.208370 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:21.208339 2562 generic.go:356] "Generic (PLEG): container finished" podID="69981e25-a1fb-4e6d-b728-8b15386fa5f6" containerID="bd07abc6b76e97fb1e817f9be5bd1944699af079d7bf58b63acc81d9a461c0bb" exitCode=0 Jul 15 17:38:21.208733 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:21.208383 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck" event={"ID":"69981e25-a1fb-4e6d-b728-8b15386fa5f6","Type":"ContainerDied","Data":"bd07abc6b76e97fb1e817f9be5bd1944699af079d7bf58b63acc81d9a461c0bb"} Jul 15 17:38:21.208733 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:21.208422 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck" event={"ID":"69981e25-a1fb-4e6d-b728-8b15386fa5f6","Type":"ContainerStarted","Data":"df28e89816b7888cf3664f01a21d35c3bb3020fce395adba33a614c5cb0a13ef"} Jul 15 17:38:22.337694 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:22.337668 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck" Jul 15 17:38:22.411078 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:22.411053 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69981e25-a1fb-4e6d-b728-8b15386fa5f6-maas-api-service-ca\") pod \"69981e25-a1fb-4e6d-b728-8b15386fa5f6\" (UID: \"69981e25-a1fb-4e6d-b728-8b15386fa5f6\") " Jul 15 17:38:22.411501 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:22.411475 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69981e25-a1fb-4e6d-b728-8b15386fa5f6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "69981e25-a1fb-4e6d-b728-8b15386fa5f6" (UID: "69981e25-a1fb-4e6d-b728-8b15386fa5f6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:38:22.512562 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:22.512484 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69981e25-a1fb-4e6d-b728-8b15386fa5f6-maas-api-service-ca\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:38:23.219354 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:23.219328 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck" Jul 15 17:38:23.219498 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:23.219321 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck" event={"ID":"69981e25-a1fb-4e6d-b728-8b15386fa5f6","Type":"ContainerDied","Data":"df28e89816b7888cf3664f01a21d35c3bb3020fce395adba33a614c5cb0a13ef"} Jul 15 17:38:23.219498 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:23.219448 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df28e89816b7888cf3664f01a21d35c3bb3020fce395adba33a614c5cb0a13ef" Jul 15 17:38:33.930489 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:33.930454 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:38:55.814346 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:38:55.814310 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:39:08.607133 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:08.607099 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:39:13.504858 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:13.504829 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d83283-vp74m"] Jul 15 17:39:13.505213 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:13.505195 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69981e25-a1fb-4e6d-b728-8b15386fa5f6" containerName="revoke-keys" Jul 15 17:39:13.505213 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:13.505208 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="69981e25-a1fb-4e6d-b728-8b15386fa5f6" containerName="revoke-keys" Jul 15 17:39:13.505321 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:13.505280 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="69981e25-a1fb-4e6d-b728-8b15386fa5f6" containerName="revoke-keys" Jul 15 17:39:13.508616 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:13.508595 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d83283-vp74m" Jul 15 17:39:13.511601 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:13.511576 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:39:13.511950 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:13.511932 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:39:13.519573 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:13.519553 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d83283-vp74m"] Jul 15 17:39:13.560626 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:13.560601 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/830c1568-3bf1-47d1-9d60-303cef159ed0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d83283-vp74m\" (UID: \"830c1568-3bf1-47d1-9d60-303cef159ed0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d83283-vp74m" Jul 15 17:39:13.661179 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:13.661153 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/830c1568-3bf1-47d1-9d60-303cef159ed0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d83283-vp74m\" (UID: \"830c1568-3bf1-47d1-9d60-303cef159ed0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d83283-vp74m" Jul 15 17:39:13.661733 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:13.661715 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/830c1568-3bf1-47d1-9d60-303cef159ed0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d83283-vp74m\" (UID: \"830c1568-3bf1-47d1-9d60-303cef159ed0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d83283-vp74m" Jul 15 17:39:13.820152 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:13.820075 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d83283-vp74m" Jul 15 17:39:13.960039 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:13.960015 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d83283-vp74m"] Jul 15 17:39:13.962457 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:39:13.962421 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod830c1568_3bf1_47d1_9d60_303cef159ed0.slice/crio-c26fc452fbf4113a20003eaae4c9e677e8b9f3cc601bdd5f5a867497e33e53ff WatchSource:0}: Error finding container c26fc452fbf4113a20003eaae4c9e677e8b9f3cc601bdd5f5a867497e33e53ff: Status 404 returned error can't find the container with id c26fc452fbf4113a20003eaae4c9e677e8b9f3cc601bdd5f5a867497e33e53ff Jul 15 17:39:14.457272 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:14.457185 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d83283-vp74m" event={"ID":"830c1568-3bf1-47d1-9d60-303cef159ed0","Type":"ContainerStarted","Data":"075e0a2637918a0d9e70058a9d3a2c6ed69a3e2fb212ba270402db29f9d0dd35"} Jul 15 17:39:14.457272 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:14.457243 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d83283-vp74m" event={"ID":"830c1568-3bf1-47d1-9d60-303cef159ed0","Type":"ContainerStarted","Data":"c26fc452fbf4113a20003eaae4c9e677e8b9f3cc601bdd5f5a867497e33e53ff"} Jul 15 17:39:14.476621 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:14.476572 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d83283-vp74m" podStartSLOduration=1.47655593 podStartE2EDuration="1.47655593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:39:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:39:14.474450647 +0000 UTC m=+2570.523447839" watchObservedRunningTime="2026-07-15 17:39:14.47655593 +0000 UTC m=+2570.525553176" Jul 15 17:39:15.463371 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:15.463334 2562 generic.go:356] "Generic (PLEG): container finished" podID="830c1568-3bf1-47d1-9d60-303cef159ed0" containerID="075e0a2637918a0d9e70058a9d3a2c6ed69a3e2fb212ba270402db29f9d0dd35" exitCode=0 Jul 15 17:39:15.463747 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:15.463424 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d83283-vp74m" event={"ID":"830c1568-3bf1-47d1-9d60-303cef159ed0","Type":"ContainerDied","Data":"075e0a2637918a0d9e70058a9d3a2c6ed69a3e2fb212ba270402db29f9d0dd35"} Jul 15 17:39:16.600868 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:16.600845 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d83283-vp74m" Jul 15 17:39:16.692357 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:16.692320 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/830c1568-3bf1-47d1-9d60-303cef159ed0-maas-api-service-ca\") pod \"830c1568-3bf1-47d1-9d60-303cef159ed0\" (UID: \"830c1568-3bf1-47d1-9d60-303cef159ed0\") " Jul 15 17:39:16.692688 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:16.692664 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/830c1568-3bf1-47d1-9d60-303cef159ed0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "830c1568-3bf1-47d1-9d60-303cef159ed0" (UID: "830c1568-3bf1-47d1-9d60-303cef159ed0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:39:16.794049 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:16.793947 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/830c1568-3bf1-47d1-9d60-303cef159ed0-maas-api-service-ca\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:39:17.476005 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:17.475970 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d83283-vp74m" Jul 15 17:39:17.476005 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:17.475990 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d83283-vp74m" event={"ID":"830c1568-3bf1-47d1-9d60-303cef159ed0","Type":"ContainerDied","Data":"c26fc452fbf4113a20003eaae4c9e677e8b9f3cc601bdd5f5a867497e33e53ff"} Jul 15 17:39:17.476229 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:17.476042 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c26fc452fbf4113a20003eaae4c9e677e8b9f3cc601bdd5f5a867497e33e53ff" Jul 15 17:39:27.892286 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:27.892248 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:39:39.813685 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:39.813650 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58"] Jul 15 17:39:39.814214 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:39.814063 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="830c1568-3bf1-47d1-9d60-303cef159ed0" containerName="revoke-keys" Jul 15 17:39:39.814214 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:39.814077 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="830c1568-3bf1-47d1-9d60-303cef159ed0" containerName="revoke-keys" Jul 15 17:39:39.814214 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:39.814142 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="830c1568-3bf1-47d1-9d60-303cef159ed0" containerName="revoke-keys" Jul 15 17:39:39.817020 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:39.817003 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58" Jul 15 17:39:39.819489 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:39.819468 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:39:39.819618 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:39.819496 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:39:39.823681 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:39.823647 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58"] Jul 15 17:39:39.998650 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:39.998617 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebf3f54b-8a3d-4f42-afa7-9598f63adef6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58\" (UID: \"ebf3f54b-8a3d-4f42-afa7-9598f63adef6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58" Jul 15 17:39:40.100044 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:40.099936 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebf3f54b-8a3d-4f42-afa7-9598f63adef6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58\" (UID: \"ebf3f54b-8a3d-4f42-afa7-9598f63adef6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58" Jul 15 17:39:40.100622 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:40.100602 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebf3f54b-8a3d-4f42-afa7-9598f63adef6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58\" (UID: \"ebf3f54b-8a3d-4f42-afa7-9598f63adef6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58" Jul 15 17:39:40.129026 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:40.128992 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58" Jul 15 17:39:40.265192 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:40.265169 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58"] Jul 15 17:39:40.267856 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:39:40.267827 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebf3f54b_8a3d_4f42_afa7_9598f63adef6.slice/crio-59cb1a70701f234df13f1cda1ef7ff7374aa06b97546b642411d26b144edb676 WatchSource:0}: Error finding container 59cb1a70701f234df13f1cda1ef7ff7374aa06b97546b642411d26b144edb676: Status 404 returned error can't find the container with id 59cb1a70701f234df13f1cda1ef7ff7374aa06b97546b642411d26b144edb676 Jul 15 17:39:40.583344 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:40.583301 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58" event={"ID":"ebf3f54b-8a3d-4f42-afa7-9598f63adef6","Type":"ContainerStarted","Data":"09d984117d6358fda884e4130faaa93b0cbc930368bcb51bdbc8de977b6a6189"} Jul 15 17:39:40.583502 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:40.583366 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58" event={"ID":"ebf3f54b-8a3d-4f42-afa7-9598f63adef6","Type":"ContainerStarted","Data":"59cb1a70701f234df13f1cda1ef7ff7374aa06b97546b642411d26b144edb676"} Jul 15 17:39:40.603365 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:40.603315 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58" podStartSLOduration=1.603280313 podStartE2EDuration="1.603280313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:39:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:39:40.601423503 +0000 UTC m=+2596.650420689" watchObservedRunningTime="2026-07-15 17:39:40.603280313 +0000 UTC m=+2596.652277504" Jul 15 17:39:41.590018 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:41.589981 2562 generic.go:356] "Generic (PLEG): container finished" podID="ebf3f54b-8a3d-4f42-afa7-9598f63adef6" containerID="09d984117d6358fda884e4130faaa93b0cbc930368bcb51bdbc8de977b6a6189" exitCode=0 Jul 15 17:39:41.590423 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:41.590027 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58" event={"ID":"ebf3f54b-8a3d-4f42-afa7-9598f63adef6","Type":"ContainerDied","Data":"09d984117d6358fda884e4130faaa93b0cbc930368bcb51bdbc8de977b6a6189"} Jul 15 17:39:42.741273 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:42.741237 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58" Jul 15 17:39:42.930748 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:42.930726 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebf3f54b-8a3d-4f42-afa7-9598f63adef6-maas-api-service-ca\") pod \"ebf3f54b-8a3d-4f42-afa7-9598f63adef6\" (UID: \"ebf3f54b-8a3d-4f42-afa7-9598f63adef6\") " Jul 15 17:39:42.931071 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:42.931048 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebf3f54b-8a3d-4f42-afa7-9598f63adef6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ebf3f54b-8a3d-4f42-afa7-9598f63adef6" (UID: "ebf3f54b-8a3d-4f42-afa7-9598f63adef6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:39:43.031630 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:43.031591 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebf3f54b-8a3d-4f42-afa7-9598f63adef6-maas-api-service-ca\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:39:43.600304 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:43.600259 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58" event={"ID":"ebf3f54b-8a3d-4f42-afa7-9598f63adef6","Type":"ContainerDied","Data":"59cb1a70701f234df13f1cda1ef7ff7374aa06b97546b642411d26b144edb676"} Jul 15 17:39:43.600304 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:43.600306 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58" Jul 15 17:39:43.600510 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:43.600318 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59cb1a70701f234df13f1cda1ef7ff7374aa06b97546b642411d26b144edb676" Jul 15 17:39:53.489050 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:39:53.489010 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:40:14.417050 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:14.417008 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:40:29.494715 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:29.494680 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:40:39.214506 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.214474 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx"] Jul 15 17:40:39.214859 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.214841 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebf3f54b-8a3d-4f42-afa7-9598f63adef6" containerName="revoke-keys" Jul 15 17:40:39.214859 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.214852 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebf3f54b-8a3d-4f42-afa7-9598f63adef6" containerName="revoke-keys" Jul 15 17:40:39.214928 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.214921 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebf3f54b-8a3d-4f42-afa7-9598f63adef6" containerName="revoke-keys" Jul 15 17:40:39.218015 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.217997 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.220383 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.220362 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3d6c37\"/\"kube-root-ca.crt\"" Jul 15 17:40:39.221230 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.221200 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3d6c37\"/\"auth-test-model-3d6c37-kserve-self-signed-certs\"" Jul 15 17:40:39.221230 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.221218 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3d6c37\"/\"default-dockercfg-xds9s\"" Jul 15 17:40:39.221369 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.221268 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3d6c37\"/\"openshift-service-ca.crt\"" Jul 15 17:40:39.227924 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.227901 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx"] Jul 15 17:40:39.326799 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.326763 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-tmp-dir\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.326799 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.326801 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81da08af-6481-47c2-b50c-4d303b6419a6-tls-certs\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.327012 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.326844 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-home\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.327012 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.326896 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrcsp\" (UniqueName: \"kubernetes.io/projected/81da08af-6481-47c2-b50c-4d303b6419a6-kube-api-access-hrcsp\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.327012 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.326964 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-dshm\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.327012 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.327001 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-model-cache\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.427698 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.427666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-dshm\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.427698 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.427704 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-model-cache\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.427894 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.427822 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-tmp-dir\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.427894 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.427855 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81da08af-6481-47c2-b50c-4d303b6419a6-tls-certs\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.427974 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.427938 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-home\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.428026 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.428003 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrcsp\" (UniqueName: \"kubernetes.io/projected/81da08af-6481-47c2-b50c-4d303b6419a6-kube-api-access-hrcsp\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.428091 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.428070 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-model-cache\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.428162 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.428142 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-tmp-dir\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.428337 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.428285 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-home\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.429950 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.429916 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-dshm\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.430303 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.430270 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81da08af-6481-47c2-b50c-4d303b6419a6-tls-certs\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.435923 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.435895 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrcsp\" (UniqueName: \"kubernetes.io/projected/81da08af-6481-47c2-b50c-4d303b6419a6-kube-api-access-hrcsp\") pod \"auth-test-model-3d6c37-kserve-8ffc4659f-swswx\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.527466 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.527386 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:39.656912 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.656887 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx"] Jul 15 17:40:39.658681 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:40:39.658656 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81da08af_6481_47c2_b50c_4d303b6419a6.slice/crio-49ea88be3b75e89f54ac085cc8a5db03ed52d07c52bbdd4e33688f15f3ad7be0 WatchSource:0}: Error finding container 49ea88be3b75e89f54ac085cc8a5db03ed52d07c52bbdd4e33688f15f3ad7be0: Status 404 returned error can't find the container with id 49ea88be3b75e89f54ac085cc8a5db03ed52d07c52bbdd4e33688f15f3ad7be0 Jul 15 17:40:39.878046 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.877951 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" event={"ID":"81da08af-6481-47c2-b50c-4d303b6419a6","Type":"ContainerStarted","Data":"2e36ebe2a236e57fa2a1e03ed61eb81a8b15e747b43875797c2d3684593e9851"} Jul 15 17:40:39.878046 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.877995 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" event={"ID":"81da08af-6481-47c2-b50c-4d303b6419a6","Type":"ContainerStarted","Data":"49ea88be3b75e89f54ac085cc8a5db03ed52d07c52bbdd4e33688f15f3ad7be0"} Jul 15 17:40:39.896879 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:39.896830 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" podStartSLOduration=0.896814327 podStartE2EDuration="896.814327ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:40:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:40:39.894935738 +0000 UTC m=+2655.943932930" watchObservedRunningTime="2026-07-15 17:40:39.896814327 +0000 UTC m=+2655.945811513" Jul 15 17:40:42.684330 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:42.684299 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:40:49.528088 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:49.528055 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:49.528569 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:49.528102 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:49.540438 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:49.540414 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:40:49.936809 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:40:49.936783 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:41:24.602311 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:41:24.602215 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:41:24.631632 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:41:24.631608 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:41:24.637139 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:41:24.637100 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:41:24.640410 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:41:24.640389 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:41:24.645786 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:41:24.645769 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:41:41.792184 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:41:41.792148 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:41:59.992894 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:41:59.992863 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:42:16.487649 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:42:16.487615 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:42:33.286619 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:42:33.286587 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:42:51.489821 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:42:51.489735 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:43:08.291896 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:08.291862 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:43:24.118054 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:24.118015 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx"] Jul 15 17:43:27.891544 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:27.891513 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:43:39.137134 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.137092 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" podUID="81da08af-6481-47c2-b50c-4d303b6419a6" containerName="main" containerID="cri-o://2e36ebe2a236e57fa2a1e03ed61eb81a8b15e747b43875797c2d3684593e9851" gracePeriod=45 Jul 15 17:43:39.377734 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.377706 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:43:39.439136 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.439114 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81da08af-6481-47c2-b50c-4d303b6419a6-tls-certs\") pod \"81da08af-6481-47c2-b50c-4d303b6419a6\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " Jul 15 17:43:39.439267 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.439143 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrcsp\" (UniqueName: \"kubernetes.io/projected/81da08af-6481-47c2-b50c-4d303b6419a6-kube-api-access-hrcsp\") pod \"81da08af-6481-47c2-b50c-4d303b6419a6\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " Jul 15 17:43:39.439267 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.439163 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-dshm\") pod \"81da08af-6481-47c2-b50c-4d303b6419a6\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " Jul 15 17:43:39.439267 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.439239 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-home\") pod \"81da08af-6481-47c2-b50c-4d303b6419a6\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " Jul 15 17:43:39.439267 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.439262 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-tmp-dir\") pod \"81da08af-6481-47c2-b50c-4d303b6419a6\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " Jul 15 17:43:39.439451 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.439278 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-model-cache\") pod \"81da08af-6481-47c2-b50c-4d303b6419a6\" (UID: \"81da08af-6481-47c2-b50c-4d303b6419a6\") " Jul 15 17:43:39.439536 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.439512 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "81da08af-6481-47c2-b50c-4d303b6419a6" (UID: "81da08af-6481-47c2-b50c-4d303b6419a6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:43:39.439598 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.439528 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-home" (OuterVolumeSpecName: "home") pod "81da08af-6481-47c2-b50c-4d303b6419a6" (UID: "81da08af-6481-47c2-b50c-4d303b6419a6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:43:39.439671 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.439635 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-model-cache" (OuterVolumeSpecName: "model-cache") pod "81da08af-6481-47c2-b50c-4d303b6419a6" (UID: "81da08af-6481-47c2-b50c-4d303b6419a6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:43:39.441370 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.441348 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-dshm" (OuterVolumeSpecName: "dshm") pod "81da08af-6481-47c2-b50c-4d303b6419a6" (UID: "81da08af-6481-47c2-b50c-4d303b6419a6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:43:39.441460 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.441357 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81da08af-6481-47c2-b50c-4d303b6419a6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "81da08af-6481-47c2-b50c-4d303b6419a6" (UID: "81da08af-6481-47c2-b50c-4d303b6419a6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:43:39.441460 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.441353 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81da08af-6481-47c2-b50c-4d303b6419a6-kube-api-access-hrcsp" (OuterVolumeSpecName: "kube-api-access-hrcsp") pod "81da08af-6481-47c2-b50c-4d303b6419a6" (UID: "81da08af-6481-47c2-b50c-4d303b6419a6"). InnerVolumeSpecName "kube-api-access-hrcsp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:43:39.540195 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.540171 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-home\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:43:39.540195 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.540194 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-tmp-dir\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:43:39.540360 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.540204 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-model-cache\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:43:39.540360 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.540212 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81da08af-6481-47c2-b50c-4d303b6419a6-tls-certs\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:43:39.540360 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.540220 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrcsp\" (UniqueName: \"kubernetes.io/projected/81da08af-6481-47c2-b50c-4d303b6419a6-kube-api-access-hrcsp\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:43:39.540360 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.540227 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81da08af-6481-47c2-b50c-4d303b6419a6-dshm\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:43:39.744501 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.744417 2562 generic.go:356] "Generic (PLEG): container finished" podID="81da08af-6481-47c2-b50c-4d303b6419a6" containerID="2e36ebe2a236e57fa2a1e03ed61eb81a8b15e747b43875797c2d3684593e9851" exitCode=0 Jul 15 17:43:39.744501 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.744469 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" Jul 15 17:43:39.744667 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.744498 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" event={"ID":"81da08af-6481-47c2-b50c-4d303b6419a6","Type":"ContainerDied","Data":"2e36ebe2a236e57fa2a1e03ed61eb81a8b15e747b43875797c2d3684593e9851"} Jul 15 17:43:39.744667 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.744541 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx" event={"ID":"81da08af-6481-47c2-b50c-4d303b6419a6","Type":"ContainerDied","Data":"49ea88be3b75e89f54ac085cc8a5db03ed52d07c52bbdd4e33688f15f3ad7be0"} Jul 15 17:43:39.744667 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.744562 2562 scope.go:117] "RemoveContainer" containerID="2e36ebe2a236e57fa2a1e03ed61eb81a8b15e747b43875797c2d3684593e9851" Jul 15 17:43:39.753402 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.753386 2562 scope.go:117] "RemoveContainer" containerID="2e36ebe2a236e57fa2a1e03ed61eb81a8b15e747b43875797c2d3684593e9851" Jul 15 17:43:39.753610 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:43:39.753592 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e36ebe2a236e57fa2a1e03ed61eb81a8b15e747b43875797c2d3684593e9851\": container with ID starting with 2e36ebe2a236e57fa2a1e03ed61eb81a8b15e747b43875797c2d3684593e9851 not found: ID does not exist" containerID="2e36ebe2a236e57fa2a1e03ed61eb81a8b15e747b43875797c2d3684593e9851" Jul 15 17:43:39.753648 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.753616 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e36ebe2a236e57fa2a1e03ed61eb81a8b15e747b43875797c2d3684593e9851"} err="failed to get container status \"2e36ebe2a236e57fa2a1e03ed61eb81a8b15e747b43875797c2d3684593e9851\": rpc error: code = NotFound desc = could not find container \"2e36ebe2a236e57fa2a1e03ed61eb81a8b15e747b43875797c2d3684593e9851\": container with ID starting with 2e36ebe2a236e57fa2a1e03ed61eb81a8b15e747b43875797c2d3684593e9851 not found: ID does not exist" Jul 15 17:43:39.770605 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.770576 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx"] Jul 15 17:43:39.777778 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:39.777750 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3d6c37/auth-test-model-3d6c37-kserve-8ffc4659f-swswx"] Jul 15 17:43:40.567004 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:40.566968 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81da08af-6481-47c2-b50c-4d303b6419a6" path="/var/lib/kubelet/pods/81da08af-6481-47c2-b50c-4d303b6419a6/volumes" Jul 15 17:43:44.601679 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:43:44.601645 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:44:02.204028 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:44:02.203997 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:44:26.297154 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:44:26.297115 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:44:45.000819 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:44:45.000776 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:44:58.833547 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:44:58.833514 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:45:00.137730 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.137699 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735625-dxdtl"] Jul 15 17:45:00.138154 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.138077 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81da08af-6481-47c2-b50c-4d303b6419a6" containerName="main" Jul 15 17:45:00.138154 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.138090 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="81da08af-6481-47c2-b50c-4d303b6419a6" containerName="main" Jul 15 17:45:00.138242 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.138163 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="81da08af-6481-47c2-b50c-4d303b6419a6" containerName="main" Jul 15 17:45:00.141192 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.141176 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3d6c37-29735625-4565t"] Jul 15 17:45:00.141318 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.141287 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735625-dxdtl" Jul 15 17:45:00.143648 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.143615 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:45:00.146753 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.146575 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-81a039-29735625-j2prt"] Jul 15 17:45:00.155951 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.155233 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3d6c37-29735625-4565t" Jul 15 17:45:00.161770 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.161748 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735625-dxdtl" Jul 15 17:45:00.162484 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.162463 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735625-dxdtl"] Jul 15 17:45:00.162603 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.162555 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-81a039-29735625-j2prt" Jul 15 17:45:00.162899 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.162886 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-81a039-29735625-j2prt"] Jul 15 17:45:00.167874 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.167849 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3d6c37-29735625-4565t"] Jul 15 17:45:00.172605 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.172588 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3d6c37-29735625-4565t" Jul 15 17:45:00.207224 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.207192 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-81a039-29735625-j2prt" Jul 15 17:45:00.320926 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.320869 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735625-dxdtl"] Jul 15 17:45:00.323512 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:45:00.323463 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf60acee0_9b0c_4b54_b20c_5431e329aa4e.slice/crio-23d26ec5ae1256decc681be2d9a55395db211185319989e1c48bde01e3c5faac WatchSource:0}: Error finding container 23d26ec5ae1256decc681be2d9a55395db211185319989e1c48bde01e3c5faac: Status 404 returned error can't find the container with id 23d26ec5ae1256decc681be2d9a55395db211185319989e1c48bde01e3c5faac Jul 15 17:45:00.386548 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.386523 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-81a039-29735625-j2prt"] Jul 15 17:45:00.387989 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:45:00.387938 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc1c56e_b6d7_41dd_864e_81bccf5423e1.slice/crio-0e1a0c74052346596c2feae7733a49145f337c1e0bc0e5f759ba8f504924646b WatchSource:0}: Error finding container 0e1a0c74052346596c2feae7733a49145f337c1e0bc0e5f759ba8f504924646b: Status 404 returned error can't find the container with id 0e1a0c74052346596c2feae7733a49145f337c1e0bc0e5f759ba8f504924646b Jul 15 17:45:00.550239 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:00.550218 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3d6c37-29735625-4565t"] Jul 15 17:45:00.551806 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:45:00.551782 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda14f0ad_954e_40ca_bae2_8fbc077845e4.slice/crio-1dc13fe0f369a32c61e4ff817c198b8ef82171bfbe1fdb0f6e4ca67f57476ef3 WatchSource:0}: Error finding container 1dc13fe0f369a32c61e4ff817c198b8ef82171bfbe1fdb0f6e4ca67f57476ef3: Status 404 returned error can't find the container with id 1dc13fe0f369a32c61e4ff817c198b8ef82171bfbe1fdb0f6e4ca67f57476ef3 Jul 15 17:45:01.130430 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:01.130331 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3d6c37-29735625-4565t" event={"ID":"da14f0ad-954e-40ca-bae2-8fbc077845e4","Type":"ContainerStarted","Data":"2baa350baa2704c457a7f138ba612aa91a235fcf0a8bf0494d0687af6e35b7e6"} Jul 15 17:45:01.130430 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:01.130398 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3d6c37-29735625-4565t" event={"ID":"da14f0ad-954e-40ca-bae2-8fbc077845e4","Type":"ContainerStarted","Data":"1dc13fe0f369a32c61e4ff817c198b8ef82171bfbe1fdb0f6e4ca67f57476ef3"} Jul 15 17:45:01.132142 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:01.132055 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-81a039-29735625-j2prt" event={"ID":"2bc1c56e-b6d7-41dd-864e-81bccf5423e1","Type":"ContainerStarted","Data":"6d8a4cdfe203cd4eb501cd27d3a78a4b46682216d077d817c707e2435d1344c9"} Jul 15 17:45:01.132142 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:01.132128 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-81a039-29735625-j2prt" event={"ID":"2bc1c56e-b6d7-41dd-864e-81bccf5423e1","Type":"ContainerStarted","Data":"0e1a0c74052346596c2feae7733a49145f337c1e0bc0e5f759ba8f504924646b"} Jul 15 17:45:01.133590 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:01.133563 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735625-dxdtl" event={"ID":"f60acee0-9b0c-4b54-b20c-5431e329aa4e","Type":"ContainerStarted","Data":"0950381ecfad9c0b67d4570d0167fba2563e03fcc2faa47bab0f77b32f7d9884"} Jul 15 17:45:01.133706 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:01.133610 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735625-dxdtl" event={"ID":"f60acee0-9b0c-4b54-b20c-5431e329aa4e","Type":"ContainerStarted","Data":"23d26ec5ae1256decc681be2d9a55395db211185319989e1c48bde01e3c5faac"} Jul 15 17:45:01.149725 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:01.149684 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3d6c37-29735625-4565t" podStartSLOduration=1.149670926 podStartE2EDuration="1.149670926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:45:01.147253376 +0000 UTC m=+2917.196250567" watchObservedRunningTime="2026-07-15 17:45:01.149670926 +0000 UTC m=+2917.198668117" Jul 15 17:45:01.168191 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:01.168148 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-81a039-29735625-j2prt" podStartSLOduration=1.168134732 podStartE2EDuration="1.168134732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:45:01.165819991 +0000 UTC m=+2917.214817181" watchObservedRunningTime="2026-07-15 17:45:01.168134732 +0000 UTC m=+2917.217131917" Jul 15 17:45:01.185279 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:01.185231 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735625-dxdtl" podStartSLOduration=1.185216779 podStartE2EDuration="1.185216779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:45:01.183315702 +0000 UTC m=+2917.232312893" watchObservedRunningTime="2026-07-15 17:45:01.185216779 +0000 UTC m=+2917.234213969" Jul 15 17:45:02.141126 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:02.141091 2562 generic.go:356] "Generic (PLEG): container finished" podID="f60acee0-9b0c-4b54-b20c-5431e329aa4e" containerID="0950381ecfad9c0b67d4570d0167fba2563e03fcc2faa47bab0f77b32f7d9884" exitCode=0 Jul 15 17:45:02.141340 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:02.141182 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735625-dxdtl" event={"ID":"f60acee0-9b0c-4b54-b20c-5431e329aa4e","Type":"ContainerDied","Data":"0950381ecfad9c0b67d4570d0167fba2563e03fcc2faa47bab0f77b32f7d9884"} Jul 15 17:45:02.142635 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:02.142615 2562 generic.go:356] "Generic (PLEG): container finished" podID="da14f0ad-954e-40ca-bae2-8fbc077845e4" containerID="2baa350baa2704c457a7f138ba612aa91a235fcf0a8bf0494d0687af6e35b7e6" exitCode=0 Jul 15 17:45:02.142738 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:02.142694 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3d6c37-29735625-4565t" event={"ID":"da14f0ad-954e-40ca-bae2-8fbc077845e4","Type":"ContainerDied","Data":"2baa350baa2704c457a7f138ba612aa91a235fcf0a8bf0494d0687af6e35b7e6"} Jul 15 17:45:02.144098 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:02.144080 2562 generic.go:356] "Generic (PLEG): container finished" podID="2bc1c56e-b6d7-41dd-864e-81bccf5423e1" containerID="6d8a4cdfe203cd4eb501cd27d3a78a4b46682216d077d817c707e2435d1344c9" exitCode=0 Jul 15 17:45:02.144184 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:02.144107 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-81a039-29735625-j2prt" event={"ID":"2bc1c56e-b6d7-41dd-864e-81bccf5423e1","Type":"ContainerDied","Data":"6d8a4cdfe203cd4eb501cd27d3a78a4b46682216d077d817c707e2435d1344c9"} Jul 15 17:45:03.284921 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:03.284898 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735625-dxdtl" Jul 15 17:45:03.329126 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:03.329104 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-81a039-29735625-j2prt" Jul 15 17:45:03.332261 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:03.332242 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3d6c37-29735625-4565t" Jul 15 17:45:04.155976 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:04.155943 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-81a039-29735625-j2prt" Jul 15 17:45:04.155976 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:04.155967 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-81a039-29735625-j2prt" event={"ID":"2bc1c56e-b6d7-41dd-864e-81bccf5423e1","Type":"ContainerDied","Data":"0e1a0c74052346596c2feae7733a49145f337c1e0bc0e5f759ba8f504924646b"} Jul 15 17:45:04.156225 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:04.156024 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e1a0c74052346596c2feae7733a49145f337c1e0bc0e5f759ba8f504924646b" Jul 15 17:45:04.157137 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:04.157120 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735625-dxdtl" Jul 15 17:45:04.157236 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:04.157162 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735625-dxdtl" event={"ID":"f60acee0-9b0c-4b54-b20c-5431e329aa4e","Type":"ContainerDied","Data":"23d26ec5ae1256decc681be2d9a55395db211185319989e1c48bde01e3c5faac"} Jul 15 17:45:04.157236 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:04.157200 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23d26ec5ae1256decc681be2d9a55395db211185319989e1c48bde01e3c5faac" Jul 15 17:45:04.158584 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:04.158558 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3d6c37-29735625-4565t" event={"ID":"da14f0ad-954e-40ca-bae2-8fbc077845e4","Type":"ContainerDied","Data":"1dc13fe0f369a32c61e4ff817c198b8ef82171bfbe1fdb0f6e4ca67f57476ef3"} Jul 15 17:45:04.158584 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:04.158578 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3d6c37-29735625-4565t" Jul 15 17:45:04.158724 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:04.158597 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1dc13fe0f369a32c61e4ff817c198b8ef82171bfbe1fdb0f6e4ca67f57476ef3" Jul 15 17:45:11.706509 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:11.706475 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:45:24.104375 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.104339 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr"] Jul 15 17:45:24.104743 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.104675 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f60acee0-9b0c-4b54-b20c-5431e329aa4e" containerName="cleanup" Jul 15 17:45:24.104743 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.104685 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f60acee0-9b0c-4b54-b20c-5431e329aa4e" containerName="cleanup" Jul 15 17:45:24.104743 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.104696 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da14f0ad-954e-40ca-bae2-8fbc077845e4" containerName="cleanup" Jul 15 17:45:24.104743 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.104701 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="da14f0ad-954e-40ca-bae2-8fbc077845e4" containerName="cleanup" Jul 15 17:45:24.104743 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.104720 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bc1c56e-b6d7-41dd-864e-81bccf5423e1" containerName="cleanup" Jul 15 17:45:24.104743 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.104726 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bc1c56e-b6d7-41dd-864e-81bccf5423e1" containerName="cleanup" Jul 15 17:45:24.104924 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.104778 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bc1c56e-b6d7-41dd-864e-81bccf5423e1" containerName="cleanup" Jul 15 17:45:24.104924 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.104786 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f60acee0-9b0c-4b54-b20c-5431e329aa4e" containerName="cleanup" Jul 15 17:45:24.104924 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.104794 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="da14f0ad-954e-40ca-bae2-8fbc077845e4" containerName="cleanup" Jul 15 17:45:24.108160 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.108132 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.112070 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.112044 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-81a039\"/\"kube-root-ca.crt\"" Jul 15 17:45:24.112172 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.112053 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-81a039\"/\"openshift-service-ca.crt\"" Jul 15 17:45:24.112172 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.112130 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-81a039\"/\"rate-test-model-81a039-kserve-self-signed-certs\"" Jul 15 17:45:24.112256 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.112210 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-81a039\"/\"default-dockercfg-c4jvt\"" Jul 15 17:45:24.117692 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.117670 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr"] Jul 15 17:45:24.141923 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.141896 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-tmp-dir\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.142034 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.141991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njbcb\" (UniqueName: \"kubernetes.io/projected/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-kube-api-access-njbcb\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.142034 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.142013 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-model-cache\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.142110 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.142038 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-dshm\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.142110 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.142100 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-home\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.142183 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.142133 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-tls-certs\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.243515 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.243484 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-tmp-dir\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.243686 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.243560 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njbcb\" (UniqueName: \"kubernetes.io/projected/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-kube-api-access-njbcb\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.243686 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.243583 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-model-cache\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.243686 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.243599 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-dshm\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.243686 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.243625 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-home\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.243686 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.243650 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-tls-certs\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.243972 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.243941 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-tmp-dir\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.243972 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.243960 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-model-cache\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.244118 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.244022 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-home\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.245709 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.245682 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-dshm\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.245980 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.245964 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-tls-certs\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.252796 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.252772 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njbcb\" (UniqueName: \"kubernetes.io/projected/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-kube-api-access-njbcb\") pod \"rate-test-model-81a039-kserve-6f4d564d8c-4t5zr\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.421423 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.421396 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-81a039\"/\"default-dockercfg-c4jvt\"" Jul 15 17:45:24.429439 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.429417 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:24.559989 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:24.559964 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr"] Jul 15 17:45:24.561635 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:45:24.561599 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55a9b50b_d23d_47a2_aaac_bdaefd4afe70.slice/crio-5fe7ccb106270296fa4139b84eb0c9e4d0bb6c2364e51997fc96147f9f50dba6 WatchSource:0}: Error finding container 5fe7ccb106270296fa4139b84eb0c9e4d0bb6c2364e51997fc96147f9f50dba6: Status 404 returned error can't find the container with id 5fe7ccb106270296fa4139b84eb0c9e4d0bb6c2364e51997fc96147f9f50dba6 Jul 15 17:45:25.264515 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:25.264478 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" event={"ID":"55a9b50b-d23d-47a2-aaac-bdaefd4afe70","Type":"ContainerStarted","Data":"5dfecee48d4ca16204ada4f26ce01eccd726b120c9366fd45622b20e822688cd"} Jul 15 17:45:25.264859 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:25.264527 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" event={"ID":"55a9b50b-d23d-47a2-aaac-bdaefd4afe70","Type":"ContainerStarted","Data":"5fe7ccb106270296fa4139b84eb0c9e4d0bb6c2364e51997fc96147f9f50dba6"} Jul 15 17:45:25.288476 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:25.288431 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" podStartSLOduration=1.288415894 podStartE2EDuration="1.288415894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:45:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:45:25.286342525 +0000 UTC m=+2941.335339718" watchObservedRunningTime="2026-07-15 17:45:25.288415894 +0000 UTC m=+2941.337413138" Jul 15 17:45:28.995519 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:28.995483 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:45:34.429646 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:34.429616 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:34.430001 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:34.429667 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:34.441814 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:34.441793 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:35.323216 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:35.323187 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:45:46.206150 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:45:46.206070 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:46:05.785132 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:05.785096 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:46:13.089343 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:13.089308 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr"] Jul 15 17:46:18.501750 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:18.501695 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:46:24.663006 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:24.662974 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:46:24.669415 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:24.669391 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:46:24.672119 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:24.672098 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:46:24.677857 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:24.677839 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wlpcj_e8547271-3ba8-4a2c-8e1e-d3c18de88bc2/ovn-acl-logging/0.log" Jul 15 17:46:28.109413 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.109357 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" podUID="55a9b50b-d23d-47a2-aaac-bdaefd4afe70" containerName="main" containerID="cri-o://5dfecee48d4ca16204ada4f26ce01eccd726b120c9366fd45622b20e822688cd" gracePeriod=45 Jul 15 17:46:28.355976 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.355954 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:46:28.415211 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.415182 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-home\") pod \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " Jul 15 17:46:28.415385 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.415221 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-tls-certs\") pod \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " Jul 15 17:46:28.415385 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.415274 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njbcb\" (UniqueName: \"kubernetes.io/projected/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-kube-api-access-njbcb\") pod \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " Jul 15 17:46:28.415385 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.415360 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-model-cache\") pod \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " Jul 15 17:46:28.415690 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.415395 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-dshm\") pod \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " Jul 15 17:46:28.415690 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.415477 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-tmp-dir\") pod \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\" (UID: \"55a9b50b-d23d-47a2-aaac-bdaefd4afe70\") " Jul 15 17:46:28.415690 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.415472 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-home" (OuterVolumeSpecName: "home") pod "55a9b50b-d23d-47a2-aaac-bdaefd4afe70" (UID: "55a9b50b-d23d-47a2-aaac-bdaefd4afe70"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:28.415690 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.415639 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-model-cache" (OuterVolumeSpecName: "model-cache") pod "55a9b50b-d23d-47a2-aaac-bdaefd4afe70" (UID: "55a9b50b-d23d-47a2-aaac-bdaefd4afe70"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:28.415875 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.415768 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "55a9b50b-d23d-47a2-aaac-bdaefd4afe70" (UID: "55a9b50b-d23d-47a2-aaac-bdaefd4afe70"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:28.415875 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.415790 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-model-cache\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:46:28.415875 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.415807 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-home\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:46:28.417475 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.417444 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-dshm" (OuterVolumeSpecName: "dshm") pod "55a9b50b-d23d-47a2-aaac-bdaefd4afe70" (UID: "55a9b50b-d23d-47a2-aaac-bdaefd4afe70"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:28.417575 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.417503 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "55a9b50b-d23d-47a2-aaac-bdaefd4afe70" (UID: "55a9b50b-d23d-47a2-aaac-bdaefd4afe70"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:46:28.417804 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.417788 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-kube-api-access-njbcb" (OuterVolumeSpecName: "kube-api-access-njbcb") pod "55a9b50b-d23d-47a2-aaac-bdaefd4afe70" (UID: "55a9b50b-d23d-47a2-aaac-bdaefd4afe70"). InnerVolumeSpecName "kube-api-access-njbcb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:46:28.516244 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.516215 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njbcb\" (UniqueName: \"kubernetes.io/projected/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-kube-api-access-njbcb\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:46:28.516244 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.516243 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-dshm\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:46:28.516493 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.516253 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-tmp-dir\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:46:28.516493 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.516262 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55a9b50b-d23d-47a2-aaac-bdaefd4afe70-tls-certs\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:46:28.570020 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.569984 2562 generic.go:356] "Generic (PLEG): container finished" podID="55a9b50b-d23d-47a2-aaac-bdaefd4afe70" containerID="5dfecee48d4ca16204ada4f26ce01eccd726b120c9366fd45622b20e822688cd" exitCode=0 Jul 15 17:46:28.570180 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.570040 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" Jul 15 17:46:28.570180 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.570044 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" event={"ID":"55a9b50b-d23d-47a2-aaac-bdaefd4afe70","Type":"ContainerDied","Data":"5dfecee48d4ca16204ada4f26ce01eccd726b120c9366fd45622b20e822688cd"} Jul 15 17:46:28.570180 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.570073 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr" event={"ID":"55a9b50b-d23d-47a2-aaac-bdaefd4afe70","Type":"ContainerDied","Data":"5fe7ccb106270296fa4139b84eb0c9e4d0bb6c2364e51997fc96147f9f50dba6"} Jul 15 17:46:28.570180 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.570093 2562 scope.go:117] "RemoveContainer" containerID="5dfecee48d4ca16204ada4f26ce01eccd726b120c9366fd45622b20e822688cd" Jul 15 17:46:28.580055 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.580036 2562 scope.go:117] "RemoveContainer" containerID="5dfecee48d4ca16204ada4f26ce01eccd726b120c9366fd45622b20e822688cd" Jul 15 17:46:28.580427 ip-10-0-134-118 kubenswrapper[2562]: E0715 17:46:28.580400 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dfecee48d4ca16204ada4f26ce01eccd726b120c9366fd45622b20e822688cd\": container with ID starting with 5dfecee48d4ca16204ada4f26ce01eccd726b120c9366fd45622b20e822688cd not found: ID does not exist" containerID="5dfecee48d4ca16204ada4f26ce01eccd726b120c9366fd45622b20e822688cd" Jul 15 17:46:28.580522 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.580437 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dfecee48d4ca16204ada4f26ce01eccd726b120c9366fd45622b20e822688cd"} err="failed to get container status \"5dfecee48d4ca16204ada4f26ce01eccd726b120c9366fd45622b20e822688cd\": rpc error: code = NotFound desc = could not find container \"5dfecee48d4ca16204ada4f26ce01eccd726b120c9366fd45622b20e822688cd\": container with ID starting with 5dfecee48d4ca16204ada4f26ce01eccd726b120c9366fd45622b20e822688cd not found: ID does not exist" Jul 15 17:46:28.592602 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.592582 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr"] Jul 15 17:46:28.598725 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:28.598705 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-81a039/rate-test-model-81a039-kserve-6f4d564d8c-4t5zr"] Jul 15 17:46:30.566944 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:30.566901 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55a9b50b-d23d-47a2-aaac-bdaefd4afe70" path="/var/lib/kubelet/pods/55a9b50b-d23d-47a2-aaac-bdaefd4afe70/volumes" Jul 15 17:46:32.810360 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:32.810327 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q"] Jul 15 17:46:32.810742 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:32.810724 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55a9b50b-d23d-47a2-aaac-bdaefd4afe70" containerName="main" Jul 15 17:46:32.810742 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:32.810737 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="55a9b50b-d23d-47a2-aaac-bdaefd4afe70" containerName="main" Jul 15 17:46:32.810814 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:32.810802 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="55a9b50b-d23d-47a2-aaac-bdaefd4afe70" containerName="main" Jul 15 17:46:32.815477 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:32.815459 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q" Jul 15 17:46:32.818051 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:32.818025 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:46:32.818406 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:32.818241 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:46:32.819909 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:32.819873 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q"] Jul 15 17:46:32.957235 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:32.957199 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3650479b-88e8-4f19-8f3b-e3fc61cd45da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q\" (UID: \"3650479b-88e8-4f19-8f3b-e3fc61cd45da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q" Jul 15 17:46:33.058119 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:33.058084 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3650479b-88e8-4f19-8f3b-e3fc61cd45da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q\" (UID: \"3650479b-88e8-4f19-8f3b-e3fc61cd45da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q" Jul 15 17:46:33.058750 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:33.058727 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3650479b-88e8-4f19-8f3b-e3fc61cd45da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q\" (UID: \"3650479b-88e8-4f19-8f3b-e3fc61cd45da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q" Jul 15 17:46:33.126888 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:33.126809 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q" Jul 15 17:46:33.260097 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:33.260066 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q"] Jul 15 17:46:33.262054 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:46:33.262031 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3650479b_88e8_4f19_8f3b_e3fc61cd45da.slice/crio-2956edcf0bb599ca19de391254ed1942ce3a98a5c13d52711c97bb8d02c9dbda WatchSource:0}: Error finding container 2956edcf0bb599ca19de391254ed1942ce3a98a5c13d52711c97bb8d02c9dbda: Status 404 returned error can't find the container with id 2956edcf0bb599ca19de391254ed1942ce3a98a5c13d52711c97bb8d02c9dbda Jul 15 17:46:33.598965 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:33.598926 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q" event={"ID":"3650479b-88e8-4f19-8f3b-e3fc61cd45da","Type":"ContainerStarted","Data":"555c04d7e9546945ff4c57548125f173f6f08b21eeb4f160bae5f25d47fdf152"} Jul 15 17:46:33.599142 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:33.598987 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q" event={"ID":"3650479b-88e8-4f19-8f3b-e3fc61cd45da","Type":"ContainerStarted","Data":"2956edcf0bb599ca19de391254ed1942ce3a98a5c13d52711c97bb8d02c9dbda"} Jul 15 17:46:33.617010 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:33.616959 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q" podStartSLOduration=1.616944095 podStartE2EDuration="1.616944095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:46:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:46:33.614859605 +0000 UTC m=+3009.663856812" watchObservedRunningTime="2026-07-15 17:46:33.616944095 +0000 UTC m=+3009.665941286" Jul 15 17:46:34.604758 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:34.604722 2562 generic.go:356] "Generic (PLEG): container finished" podID="3650479b-88e8-4f19-8f3b-e3fc61cd45da" containerID="555c04d7e9546945ff4c57548125f173f6f08b21eeb4f160bae5f25d47fdf152" exitCode=0 Jul 15 17:46:34.605150 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:34.604791 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q" event={"ID":"3650479b-88e8-4f19-8f3b-e3fc61cd45da","Type":"ContainerDied","Data":"555c04d7e9546945ff4c57548125f173f6f08b21eeb4f160bae5f25d47fdf152"} Jul 15 17:46:35.741875 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:35.741852 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q" Jul 15 17:46:35.886240 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:35.886152 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3650479b-88e8-4f19-8f3b-e3fc61cd45da-maas-api-service-ca\") pod \"3650479b-88e8-4f19-8f3b-e3fc61cd45da\" (UID: \"3650479b-88e8-4f19-8f3b-e3fc61cd45da\") " Jul 15 17:46:35.886560 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:35.886535 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3650479b-88e8-4f19-8f3b-e3fc61cd45da-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3650479b-88e8-4f19-8f3b-e3fc61cd45da" (UID: "3650479b-88e8-4f19-8f3b-e3fc61cd45da"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:46:35.987513 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:35.987477 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3650479b-88e8-4f19-8f3b-e3fc61cd45da-maas-api-service-ca\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:46:36.615226 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:36.615197 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q" event={"ID":"3650479b-88e8-4f19-8f3b-e3fc61cd45da","Type":"ContainerDied","Data":"2956edcf0bb599ca19de391254ed1942ce3a98a5c13d52711c97bb8d02c9dbda"} Jul 15 17:46:36.615226 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:36.615221 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q" Jul 15 17:46:36.615454 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:36.615250 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2956edcf0bb599ca19de391254ed1942ce3a98a5c13d52711c97bb8d02c9dbda" Jul 15 17:46:43.464687 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:43.464654 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3d6c37-29735625-4565t"] Jul 15 17:46:43.469276 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:43.469247 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3d6c37-29735625-4565t"] Jul 15 17:46:44.566908 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:44.566871 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da14f0ad-954e-40ca-bae2-8fbc077845e4" path="/var/lib/kubelet/pods/da14f0ad-954e-40ca-bae2-8fbc077845e4/volumes" Jul 15 17:46:46.701372 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:46.701343 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:46:58.989891 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:58.989857 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6"] Jul 15 17:46:58.990507 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:58.990414 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3650479b-88e8-4f19-8f3b-e3fc61cd45da" containerName="revoke-keys" Jul 15 17:46:58.990507 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:58.990435 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3650479b-88e8-4f19-8f3b-e3fc61cd45da" containerName="revoke-keys" Jul 15 17:46:58.990638 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:58.990545 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3650479b-88e8-4f19-8f3b-e3fc61cd45da" containerName="revoke-keys" Jul 15 17:46:58.994711 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:58.994689 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6" Jul 15 17:46:58.997579 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:58.997534 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dlhgl\"" Jul 15 17:46:58.997698 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:58.997653 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 17:46:59.001616 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:59.001590 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6"] Jul 15 17:46:59.075720 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:59.075682 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/113bb61f-739d-4c6b-ab1c-df3eda019146-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6\" (UID: \"113bb61f-739d-4c6b-ab1c-df3eda019146\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6" Jul 15 17:46:59.177003 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:59.176970 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/113bb61f-739d-4c6b-ab1c-df3eda019146-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6\" (UID: \"113bb61f-739d-4c6b-ab1c-df3eda019146\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6" Jul 15 17:46:59.177601 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:59.177581 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/113bb61f-739d-4c6b-ab1c-df3eda019146-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6\" (UID: \"113bb61f-739d-4c6b-ab1c-df3eda019146\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6" Jul 15 17:46:59.305501 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:59.305415 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6" Jul 15 17:46:59.438704 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:59.438680 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6"] Jul 15 17:46:59.440729 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:46:59.440701 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod113bb61f_739d_4c6b_ab1c_df3eda019146.slice/crio-3a2db8a3af74c4fc94fb8f8387cfc0051e0c379ede6042ddbe75306eae0a9594 WatchSource:0}: Error finding container 3a2db8a3af74c4fc94fb8f8387cfc0051e0c379ede6042ddbe75306eae0a9594: Status 404 returned error can't find the container with id 3a2db8a3af74c4fc94fb8f8387cfc0051e0c379ede6042ddbe75306eae0a9594 Jul 15 17:46:59.730121 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:59.730091 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6" event={"ID":"113bb61f-739d-4c6b-ab1c-df3eda019146","Type":"ContainerStarted","Data":"3617845dbf51988e210e50c9644b9ced703d2a5dc8cb5fd918ae6d249bf717de"} Jul 15 17:46:59.730265 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:59.730138 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6" event={"ID":"113bb61f-739d-4c6b-ab1c-df3eda019146","Type":"ContainerStarted","Data":"3a2db8a3af74c4fc94fb8f8387cfc0051e0c379ede6042ddbe75306eae0a9594"} Jul 15 17:46:59.750511 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:46:59.750466 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6" podStartSLOduration=1.750451198 podStartE2EDuration="1.750451198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:46:59.747866899 +0000 UTC m=+3035.796864087" watchObservedRunningTime="2026-07-15 17:46:59.750451198 +0000 UTC m=+3035.799448431" Jul 15 17:47:00.737103 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:00.737067 2562 generic.go:356] "Generic (PLEG): container finished" podID="113bb61f-739d-4c6b-ab1c-df3eda019146" containerID="3617845dbf51988e210e50c9644b9ced703d2a5dc8cb5fd918ae6d249bf717de" exitCode=0 Jul 15 17:47:00.737558 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:00.737185 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6" event={"ID":"113bb61f-739d-4c6b-ab1c-df3eda019146","Type":"ContainerDied","Data":"3617845dbf51988e210e50c9644b9ced703d2a5dc8cb5fd918ae6d249bf717de"} Jul 15 17:47:01.895563 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:01.895537 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6" Jul 15 17:47:01.898666 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:01.898645 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/113bb61f-739d-4c6b-ab1c-df3eda019146-maas-api-service-ca\") pod \"113bb61f-739d-4c6b-ab1c-df3eda019146\" (UID: \"113bb61f-739d-4c6b-ab1c-df3eda019146\") " Jul 15 17:47:01.899013 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:01.898993 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/113bb61f-739d-4c6b-ab1c-df3eda019146-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "113bb61f-739d-4c6b-ab1c-df3eda019146" (UID: "113bb61f-739d-4c6b-ab1c-df3eda019146"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:47:01.999635 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:01.999548 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/113bb61f-739d-4c6b-ab1c-df3eda019146-maas-api-service-ca\") on node \"ip-10-0-134-118.ec2.internal\" DevicePath \"\"" Jul 15 17:47:02.749083 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:02.749057 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6" Jul 15 17:47:02.749083 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:02.749070 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6" event={"ID":"113bb61f-739d-4c6b-ab1c-df3eda019146","Type":"ContainerDied","Data":"3a2db8a3af74c4fc94fb8f8387cfc0051e0c379ede6042ddbe75306eae0a9594"} Jul 15 17:47:02.749283 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:02.749117 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a2db8a3af74c4fc94fb8f8387cfc0051e0c379ede6042ddbe75306eae0a9594" Jul 15 17:47:09.712200 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:09.712167 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-81a039-29735625-j2prt"] Jul 15 17:47:09.715982 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:09.715955 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-81a039-29735625-j2prt"] Jul 15 17:47:10.566426 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:10.566394 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bc1c56e-b6d7-41dd-864e-81bccf5423e1" path="/var/lib/kubelet/pods/2bc1c56e-b6d7-41dd-864e-81bccf5423e1/volumes" Jul 15 17:47:12.907595 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:12.907559 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f7r67"] Jul 15 17:47:31.633798 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:31.633771 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b767d9f5c-klfxf_a240661c-051d-4ee2-840a-9b6ace94b96a/copy-manifests/0.log" Jul 15 17:47:31.641227 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:31.641184 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b767d9f5c-klfxf_a240661c-051d-4ee2-840a-9b6ace94b96a/manager/0.log" Jul 15 17:47:32.531207 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:32.531177 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735610-vm6fk_cbf5c92d-e0a4-4eda-9a0e-004c41e261b8/cleanup/0.log" Jul 15 17:47:32.644708 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:32.644684 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735625-dxdtl_f60acee0-9b0c-4b54-b20c-5431e329aa4e/cleanup/0.log" Jul 15 17:47:32.988223 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:32.988190 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-d83283-vp74m_830c1568-3bf1-47d1-9d60-303cef159ed0/revoke-keys/0.log" Jul 15 17:47:33.103891 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:33.103860 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-1da9d9-jmm58_ebf3f54b-8a3d-4f42-afa7-9598f63adef6/revoke-keys/0.log" Jul 15 17:47:33.218586 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:33.218553 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-65f681ff-ntcr2_db3848df-93ec-4fb6-a527-d0392839ba93/revoke-keys/0.log" Jul 15 17:47:33.353047 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:33.352970 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-b525cf-rwvkn_e1c1e4d8-503f-4971-a3db-289f81090a1c/revoke-keys/0.log" Jul 15 17:47:33.466427 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:33.466402 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-ed384a-2rwck_69981e25-a1fb-4e6d-b728-8b15386fa5f6/revoke-keys/0.log" Jul 15 17:47:33.580067 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:33.580044 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-2a15ad60-gc5tk_82a12c9d-5cb9-4dac-aed2-57980169c6e7/revoke-keys/1.log" Jul 15 17:47:33.687199 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:33.687172 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-427b7c64-j2m7r_0572ad09-7829-49f1-b246-b623f504a2bd/revoke-keys/0.log" Jul 15 17:47:33.918097 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:33.918068 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-3d6c37-l9s6q_3650479b-88e8-4f19-8f3b-e3fc61cd45da/revoke-keys/0.log" Jul 15 17:47:34.030832 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:34.030760 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-81a039-zr9r6_113bb61f-739d-4c6b-ab1c-df3eda019146/revoke-keys/0.log" Jul 15 17:47:36.228198 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:36.228167 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-f7r67_61667b9c-dd9c-453b-8829-85616cd057e9/limitador/0.log" Jul 15 17:47:36.830340 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:36.830312 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-bcq64_35f6e8b7-5a41-4696-9e24-a76d2eb42cb3/discovery/0.log" Jul 15 17:47:37.537387 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:37.537359 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68ddcc8458-j29rh_afcb1c98-744f-4fbe-8396-3217c0d7f2ba/router/0.log" Jul 15 17:47:38.003518 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:38.003487 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-swddx_a79e1a1c-d481-4036-8e89-7f37e22733ff/main/0.log" Jul 15 17:47:38.361381 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:38.361305 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-z7qv2_09f74917-92f1-4535-b5e7-86eb9389555a/main/0.log" Jul 15 17:47:46.826382 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:46.826350 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8xlk5_1356768b-d3b0-4d99-98c7-42a27e4d0818/global-pull-secret-syncer/0.log" Jul 15 17:47:47.014782 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:47.014753 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-v9jrp_abe0c218-237b-4de9-88bb-9c2623db9bfa/konnectivity-agent/0.log" Jul 15 17:47:47.069203 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:47.069177 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-118.ec2.internal_9863c9ecbe102ca15435dceaa71ff34e/haproxy/0.log" Jul 15 17:47:53.167237 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:53.167213 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-f7r67_61667b9c-dd9c-453b-8829-85616cd057e9/limitador/0.log" Jul 15 17:47:55.057274 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.057195 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-w8mhz_171694b4-df85-4c8a-9449-88aad1eeca61/kube-state-metrics/0.log" Jul 15 17:47:55.076722 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.076697 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-w8mhz_171694b4-df85-4c8a-9449-88aad1eeca61/kube-rbac-proxy-main/0.log" Jul 15 17:47:55.099764 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.099742 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-w8mhz_171694b4-df85-4c8a-9449-88aad1eeca61/kube-rbac-proxy-self/0.log" Jul 15 17:47:55.138965 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.138921 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-56bbd4b4f6-ckt7t_b4e8d0b3-8cdf-4f67-9317-b2ac533f3d4b/metrics-server/0.log" Jul 15 17:47:55.367856 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.367786 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9cbdd_b560a914-ac4b-4113-a075-1ac85ec8f68a/node-exporter/0.log" Jul 15 17:47:55.388151 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.388130 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9cbdd_b560a914-ac4b-4113-a075-1ac85ec8f68a/kube-rbac-proxy/0.log" Jul 15 17:47:55.412536 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.412518 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9cbdd_b560a914-ac4b-4113-a075-1ac85ec8f68a/init-textfile/0.log" Jul 15 17:47:55.441671 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.441648 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9h5nq_520590e3-2315-46d9-ab1a-6cfdfc36fcea/kube-rbac-proxy-main/0.log" Jul 15 17:47:55.463317 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.463281 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9h5nq_520590e3-2315-46d9-ab1a-6cfdfc36fcea/kube-rbac-proxy-self/0.log" Jul 15 17:47:55.483989 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.483968 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9h5nq_520590e3-2315-46d9-ab1a-6cfdfc36fcea/openshift-state-metrics/0.log" Jul 15 17:47:55.521517 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.521501 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe/prometheus/0.log" Jul 15 17:47:55.542171 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.542151 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe/config-reloader/0.log" Jul 15 17:47:55.569750 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.569735 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe/thanos-sidecar/0.log" Jul 15 17:47:55.597572 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.597551 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe/kube-rbac-proxy-web/0.log" Jul 15 17:47:55.626219 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.626165 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe/kube-rbac-proxy/0.log" Jul 15 17:47:55.653799 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.653783 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe/kube-rbac-proxy-thanos/0.log" Jul 15 17:47:55.689047 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.689030 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d1f02325-67bf-4b5b-9e7c-a5f1be27a9fe/init-config-reloader/0.log" Jul 15 17:47:55.904793 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.904715 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-666f98849c-58htg_bb1bfc1d-a8b7-4198-b629-73b6ae489e88/thanos-query/0.log" Jul 15 17:47:55.930166 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.930138 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-666f98849c-58htg_bb1bfc1d-a8b7-4198-b629-73b6ae489e88/kube-rbac-proxy-web/0.log" Jul 15 17:47:55.952208 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.952187 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-666f98849c-58htg_bb1bfc1d-a8b7-4198-b629-73b6ae489e88/kube-rbac-proxy/0.log" Jul 15 17:47:55.972604 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.972571 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-666f98849c-58htg_bb1bfc1d-a8b7-4198-b629-73b6ae489e88/prom-label-proxy/0.log" Jul 15 17:47:55.994869 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:55.994848 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-666f98849c-58htg_bb1bfc1d-a8b7-4198-b629-73b6ae489e88/kube-rbac-proxy-rules/0.log" Jul 15 17:47:56.018429 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.018407 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-666f98849c-58htg_bb1bfc1d-a8b7-4198-b629-73b6ae489e88/kube-rbac-proxy-metrics/0.log" Jul 15 17:47:56.582712 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.582680 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x"] Jul 15 17:47:56.583081 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.583022 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113bb61f-739d-4c6b-ab1c-df3eda019146" containerName="revoke-keys" Jul 15 17:47:56.583081 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.583033 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="113bb61f-739d-4c6b-ab1c-df3eda019146" containerName="revoke-keys" Jul 15 17:47:56.583153 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.583092 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="113bb61f-739d-4c6b-ab1c-df3eda019146" containerName="revoke-keys" Jul 15 17:47:56.586506 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.586484 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:56.589326 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.589285 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8cbcq\"/\"kube-root-ca.crt\"" Jul 15 17:47:56.589452 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.589370 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8cbcq\"/\"openshift-service-ca.crt\"" Jul 15 17:47:56.590262 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.590247 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8cbcq\"/\"default-dockercfg-zrwf4\"" Jul 15 17:47:56.604494 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.604468 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x"] Jul 15 17:47:56.668922 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.668883 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fef535c7-c555-427a-a9d4-66e46edcf9ab-proc\") pod \"perf-node-gather-daemonset-kv48x\" (UID: \"fef535c7-c555-427a-a9d4-66e46edcf9ab\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:56.669072 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.668942 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fef535c7-c555-427a-a9d4-66e46edcf9ab-podres\") pod \"perf-node-gather-daemonset-kv48x\" (UID: \"fef535c7-c555-427a-a9d4-66e46edcf9ab\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:56.669072 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.668971 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fef535c7-c555-427a-a9d4-66e46edcf9ab-lib-modules\") pod \"perf-node-gather-daemonset-kv48x\" (UID: \"fef535c7-c555-427a-a9d4-66e46edcf9ab\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:56.669072 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.669002 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5hxw\" (UniqueName: \"kubernetes.io/projected/fef535c7-c555-427a-a9d4-66e46edcf9ab-kube-api-access-f5hxw\") pod \"perf-node-gather-daemonset-kv48x\" (UID: \"fef535c7-c555-427a-a9d4-66e46edcf9ab\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:56.669072 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.669022 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fef535c7-c555-427a-a9d4-66e46edcf9ab-sys\") pod \"perf-node-gather-daemonset-kv48x\" (UID: \"fef535c7-c555-427a-a9d4-66e46edcf9ab\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:56.770105 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.770083 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fef535c7-c555-427a-a9d4-66e46edcf9ab-podres\") pod \"perf-node-gather-daemonset-kv48x\" (UID: \"fef535c7-c555-427a-a9d4-66e46edcf9ab\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:56.770259 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.770113 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fef535c7-c555-427a-a9d4-66e46edcf9ab-lib-modules\") pod \"perf-node-gather-daemonset-kv48x\" (UID: \"fef535c7-c555-427a-a9d4-66e46edcf9ab\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:56.770259 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.770135 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5hxw\" (UniqueName: \"kubernetes.io/projected/fef535c7-c555-427a-a9d4-66e46edcf9ab-kube-api-access-f5hxw\") pod \"perf-node-gather-daemonset-kv48x\" (UID: \"fef535c7-c555-427a-a9d4-66e46edcf9ab\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:56.770259 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.770201 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fef535c7-c555-427a-a9d4-66e46edcf9ab-sys\") pod \"perf-node-gather-daemonset-kv48x\" (UID: \"fef535c7-c555-427a-a9d4-66e46edcf9ab\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:56.770259 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.770253 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fef535c7-c555-427a-a9d4-66e46edcf9ab-podres\") pod \"perf-node-gather-daemonset-kv48x\" (UID: \"fef535c7-c555-427a-a9d4-66e46edcf9ab\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:56.770451 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.770263 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fef535c7-c555-427a-a9d4-66e46edcf9ab-lib-modules\") pod \"perf-node-gather-daemonset-kv48x\" (UID: \"fef535c7-c555-427a-a9d4-66e46edcf9ab\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:56.770451 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.770321 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fef535c7-c555-427a-a9d4-66e46edcf9ab-sys\") pod \"perf-node-gather-daemonset-kv48x\" (UID: \"fef535c7-c555-427a-a9d4-66e46edcf9ab\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:56.770451 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.770350 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fef535c7-c555-427a-a9d4-66e46edcf9ab-proc\") pod \"perf-node-gather-daemonset-kv48x\" (UID: \"fef535c7-c555-427a-a9d4-66e46edcf9ab\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:56.770451 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.770421 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fef535c7-c555-427a-a9d4-66e46edcf9ab-proc\") pod \"perf-node-gather-daemonset-kv48x\" (UID: \"fef535c7-c555-427a-a9d4-66e46edcf9ab\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:56.780035 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.780015 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5hxw\" (UniqueName: \"kubernetes.io/projected/fef535c7-c555-427a-a9d4-66e46edcf9ab-kube-api-access-f5hxw\") pod \"perf-node-gather-daemonset-kv48x\" (UID: \"fef535c7-c555-427a-a9d4-66e46edcf9ab\") " pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:56.896607 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:56.896550 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:57.025916 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:57.025715 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x"] Jul 15 17:47:57.028559 ip-10-0-134-118 kubenswrapper[2562]: W0715 17:47:57.028535 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfef535c7_c555_427a_a9d4_66e46edcf9ab.slice/crio-7db8c4a771f2d9d86543a8380937aa595dd30a1a9e345c7e948ae952c1d97bf1 WatchSource:0}: Error finding container 7db8c4a771f2d9d86543a8380937aa595dd30a1a9e345c7e948ae952c1d97bf1: Status 404 returned error can't find the container with id 7db8c4a771f2d9d86543a8380937aa595dd30a1a9e345c7e948ae952c1d97bf1 Jul 15 17:47:57.706845 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:57.706812 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/1.log" Jul 15 17:47:57.711929 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:57.711908 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-djhfs_a7bb56c5-b728-49f2-a7d8-e869c083e49f/console-operator/2.log" Jul 15 17:47:58.033783 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:58.033690 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" event={"ID":"fef535c7-c555-427a-a9d4-66e46edcf9ab","Type":"ContainerStarted","Data":"bfa1f681808408a6a0f05cca741f57b7525254c261ec75684d81b13efbacd81c"} Jul 15 17:47:58.033783 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:58.033748 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" event={"ID":"fef535c7-c555-427a-a9d4-66e46edcf9ab","Type":"ContainerStarted","Data":"7db8c4a771f2d9d86543a8380937aa595dd30a1a9e345c7e948ae952c1d97bf1"} Jul 15 17:47:58.034025 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:58.033842 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:47:58.052673 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:58.052626 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" podStartSLOduration=2.052613667 podStartE2EDuration="2.052613667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 17:47:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:47:58.050347837 +0000 UTC m=+3094.099345031" watchObservedRunningTime="2026-07-15 17:47:58.052613667 +0000 UTC m=+3094.101610860" Jul 15 17:47:59.840922 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:59.840892 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dph8q_65c4ae32-87b7-46f2-8370-510449c1020e/dns/0.log" Jul 15 17:47:59.867623 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:47:59.867597 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dph8q_65c4ae32-87b7-46f2-8370-510449c1020e/kube-rbac-proxy/0.log" Jul 15 17:48:00.034971 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:00.034946 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tzkgv_cc6a2e63-80e9-44d9-9708-dc85bd571957/dns-node-resolver/0.log" Jul 15 17:48:00.591834 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:00.591806 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-67748f5cf7-2m5lh_f2049531-ac65-45a6-89d4-fe638debd103/registry/0.log" Jul 15 17:48:00.638646 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:00.638619 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8w7fh_1fd95390-8b40-475c-ab93-ab641b315e09/node-ca/0.log" Jul 15 17:48:01.740957 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:01.740931 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-bcq64_35f6e8b7-5a41-4696-9e24-a76d2eb42cb3/discovery/0.log" Jul 15 17:48:01.928699 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:01.928675 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68ddcc8458-j29rh_afcb1c98-744f-4fbe-8396-3217c0d7f2ba/router/0.log" Jul 15 17:48:02.647130 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:02.647098 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jd6qq_f5d75ab3-c94f-4349-953a-05abad93f858/serve-healthcheck-canary/0.log" Jul 15 17:48:03.361163 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:03.361133 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vz9xc_5c38aa01-993a-4d62-aeaa-87020199fca1/kube-rbac-proxy/0.log" Jul 15 17:48:03.380041 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:03.380012 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vz9xc_5c38aa01-993a-4d62-aeaa-87020199fca1/exporter/0.log" Jul 15 17:48:03.400330 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:03.400312 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vz9xc_5c38aa01-993a-4d62-aeaa-87020199fca1/extractor/0.log" Jul 15 17:48:04.048465 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:04.048439 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8cbcq/perf-node-gather-daemonset-kv48x" Jul 15 17:48:05.907890 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:05.907859 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b767d9f5c-klfxf_a240661c-051d-4ee2-840a-9b6ace94b96a/manager/0.log" Jul 15 17:48:05.929773 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:05.929748 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b767d9f5c-klfxf_a240661c-051d-4ee2-840a-9b6ace94b96a/copy-manifests/0.log" Jul 15 17:48:07.442481 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:07.442455 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-bcq64_35f6e8b7-5a41-4696-9e24-a76d2eb42cb3/discovery/0.log" Jul 15 17:48:07.581774 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:07.581746 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68ddcc8458-j29rh_afcb1c98-744f-4fbe-8396-3217c0d7f2ba/router/0.log" Jul 15 17:48:07.592482 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:07.592459 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5c485d79f6-zhw88_cacf5e90-db63-4e20-a351-8f5aac61e289/manager/0.log" Jul 15 17:48:08.834006 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:08.833977 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-bcq64_35f6e8b7-5a41-4696-9e24-a76d2eb42cb3/discovery/0.log" Jul 15 17:48:08.977083 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:08.977060 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68ddcc8458-j29rh_afcb1c98-744f-4fbe-8396-3217c0d7f2ba/router/0.log" Jul 15 17:48:13.043259 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:13.043222 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-27gvz_c7ffc33d-460d-437f-baf0-c93e7efc4dff/kube-storage-version-migrator-operator/1.log" Jul 15 17:48:13.043843 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:13.043828 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-27gvz_c7ffc33d-460d-437f-baf0-c93e7efc4dff/kube-storage-version-migrator-operator/0.log" Jul 15 17:48:13.983603 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:13.983570 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6cqnq_375d7469-6ffd-4be1-94bf-2782015de14d/kube-multus/0.log" Jul 15 17:48:14.049905 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:14.049884 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dx7t2_76a7ad68-9351-4709-b119-ac5a58ca509e/kube-multus-additional-cni-plugins/0.log" Jul 15 17:48:14.071556 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:14.071489 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dx7t2_76a7ad68-9351-4709-b119-ac5a58ca509e/egress-router-binary-copy/0.log" Jul 15 17:48:14.090919 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:14.090894 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dx7t2_76a7ad68-9351-4709-b119-ac5a58ca509e/cni-plugins/0.log" Jul 15 17:48:14.111581 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:14.111560 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dx7t2_76a7ad68-9351-4709-b119-ac5a58ca509e/bond-cni-plugin/0.log" Jul 15 17:48:14.135184 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:14.135167 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dx7t2_76a7ad68-9351-4709-b119-ac5a58ca509e/routeoverride-cni/0.log" Jul 15 17:48:14.154918 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:14.154900 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dx7t2_76a7ad68-9351-4709-b119-ac5a58ca509e/whereabouts-cni-bincopy/0.log" Jul 15 17:48:14.174372 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:14.174350 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dx7t2_76a7ad68-9351-4709-b119-ac5a58ca509e/whereabouts-cni/0.log" Jul 15 17:48:14.611234 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:14.611208 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mbqkc_4f43f08e-aa70-42d2-9207-93156d956d7f/network-metrics-daemon/0.log" Jul 15 17:48:14.628672 ip-10-0-134-118 kubenswrapper[2562]: I0715 17:48:14.628652 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mbqkc_4f43f08e-aa70-42d2-9207-93156d956d7f/kube-rbac-proxy/0.log"