Sep 09 20:48:31.224305 ip-10-0-139-83 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 09 20:48:31.224319 ip-10-0-139-83 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 09 20:48:31.224326 ip-10-0-139-83 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 09 20:48:31.224541 ip-10-0-139-83 systemd[1]: Failed to start Kubernetes Kubelet. Sep 09 20:48:41.431710 ip-10-0-139-83 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 09 20:48:41.431731 ip-10-0-139-83 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 1b249e239e4d4254adaaeb856999ae58 -- Sep 09 20:50:23.055573 ip-10-0-139-83 systemd[1]: Starting Kubernetes Kubelet... Sep 09 20:50:23.401646 ip-10-0-139-83 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 20:50:23.401646 ip-10-0-139-83 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 09 20:50:23.401646 ip-10-0-139-83 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 20:50:23.401646 ip-10-0-139-83 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 09 20:50:23.401646 ip-10-0-139-83 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 20:50:23.403900 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.403815 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 09 20:50:23.409401 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409369 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 20:50:23.409401 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409391 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 20:50:23.409401 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409397 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 20:50:23.409401 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409401 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 20:50:23.409401 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409404 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 20:50:23.409401 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409408 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 20:50:23.409401 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409411 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 20:50:23.409401 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409413 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409416 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409419 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409421 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409424 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409426 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409429 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409431 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409434 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409436 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409439 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409449 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409452 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409454 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409457 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409459 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409462 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409464 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409467 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409469 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 20:50:23.409725 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409472 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409475 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409477 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409480 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409482 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409485 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409487 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409490 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409492 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409494 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409497 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409499 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409502 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409505 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409507 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409510 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409512 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409515 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409517 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 20:50:23.410207 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409520 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409523 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409527 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409531 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409535 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409537 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409540 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409542 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409545 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409548 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409555 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409557 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409560 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409562 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409565 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409568 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409571 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409573 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409576 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 20:50:23.410703 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409578 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409582 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409586 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409588 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409591 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409593 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409597 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409600 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409602 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409605 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409607 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409610 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409613 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409615 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409617 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409620 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409622 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409625 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409628 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409630 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 20:50:23.411184 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409632 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409635 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409637 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409640 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409642 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409644 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409647 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409650 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409652 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409655 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409657 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409659 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409662 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.409664 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410382 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410389 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410392 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410395 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410398 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410401 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 20:50:23.411715 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410404 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410406 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410409 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410411 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410414 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410416 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410421 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410424 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410428 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410431 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410434 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410437 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410440 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410443 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410446 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410448 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410451 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410453 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410457 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 20:50:23.412209 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410461 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410463 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410466 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410468 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410471 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410473 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410476 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410478 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410480 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410483 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410486 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410489 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410491 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410493 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410496 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410500 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410503 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410505 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410508 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410511 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 20:50:23.412741 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410513 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410516 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410518 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410521 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410524 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410527 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410529 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410533 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410535 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410537 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410540 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410543 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410545 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410548 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410550 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410552 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410555 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410558 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410560 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410562 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 20:50:23.413230 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410564 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410568 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410571 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410574 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410577 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410580 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410582 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410585 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410588 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410590 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410593 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410595 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410598 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410600 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410603 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410606 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410608 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410611 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410614 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 20:50:23.413751 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410617 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410619 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410621 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410624 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410626 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410629 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410632 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410634 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410636 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410639 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410641 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410643 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410646 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410648 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.410651 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411043 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411048 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411050 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411054 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 20:50:23.414216 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411058 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411061 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411063 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411066 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411068 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411072 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411074 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411076 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411079 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411081 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411084 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411087 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411090 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411093 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411095 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411097 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411100 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411102 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411105 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411107 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 20:50:23.414701 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411110 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411112 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411114 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411117 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411119 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411122 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411124 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411129 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411131 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411134 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411137 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411140 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411142 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411145 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411147 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411150 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411152 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411155 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411157 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411160 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411163 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 20:50:23.415191 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411166 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411169 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411171 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411174 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411177 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411179 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411182 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411184 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411187 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411190 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411192 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411195 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411197 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411200 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411202 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411205 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411207 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411210 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411213 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 20:50:23.415727 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411216 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411218 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411221 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411225 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411228 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411231 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411252 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411257 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411261 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411264 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411268 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411270 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411275 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411278 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411280 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411283 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411285 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411287 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 20:50:23.416224 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411290 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411293 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411295 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411298 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411300 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411303 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411306 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411308 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411311 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411313 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411316 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411318 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411321 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411324 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411327 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411330 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.411332 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411407 2562 flags.go:64] FLAG: --address="0.0.0.0" Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411442 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411450 2562 flags.go:64] FLAG: --anonymous-auth="true" Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411454 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 09 20:50:23.416696 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411458 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411461 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411465 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411470 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411473 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411477 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411480 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411484 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411487 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411490 2562 flags.go:64] FLAG: --cgroup-root="" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411493 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411496 2562 flags.go:64] FLAG: --client-ca-file="" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411500 2562 flags.go:64] FLAG: --cloud-provider="external" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411503 2562 flags.go:64] FLAG: --cluster-dns="[]" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411507 2562 flags.go:64] FLAG: --cluster-domain="" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411510 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411513 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411516 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411519 2562 flags.go:64] FLAG: --container-log-max-files="5" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411522 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411525 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411528 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411532 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411534 2562 flags.go:64] FLAG: --contention-profiling="false" Sep 09 20:50:23.417216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411539 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411541 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411544 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411547 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411551 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411554 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411557 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411560 2562 flags.go:64] FLAG: --enable-load-reader="false" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411563 2562 flags.go:64] FLAG: --enable-server="true" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411565 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411570 2562 flags.go:64] FLAG: --event-burst="100" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411573 2562 flags.go:64] FLAG: --event-qps="50" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411576 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411579 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411582 2562 flags.go:64] FLAG: --eviction-hard="" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411585 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411588 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411591 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411594 2562 flags.go:64] FLAG: --eviction-soft="" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411597 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411600 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411603 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411606 2562 flags.go:64] FLAG: --experimental-mounter-path="" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411609 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411611 2562 flags.go:64] FLAG: --fail-swap-on="true" Sep 09 20:50:23.417887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411614 2562 flags.go:64] FLAG: --feature-gates="" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411617 2562 flags.go:64] FLAG: --file-check-frequency="20s" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411620 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411623 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411626 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411629 2562 flags.go:64] FLAG: --healthz-port="10248" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411631 2562 flags.go:64] FLAG: --help="false" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411635 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-139-83.ec2.internal" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411640 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411643 2562 flags.go:64] FLAG: --http-check-frequency="20s" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411646 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411649 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411652 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411655 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411658 2562 flags.go:64] FLAG: --image-service-endpoint="" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411661 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411664 2562 flags.go:64] FLAG: --kube-api-burst="100" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411666 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411669 2562 flags.go:64] FLAG: --kube-api-qps="50" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411672 2562 flags.go:64] FLAG: --kube-reserved="" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411675 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411678 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411680 2562 flags.go:64] FLAG: --kubelet-cgroups="" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411683 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 09 20:50:23.418525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411686 2562 flags.go:64] FLAG: --lock-file="" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411689 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411691 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411695 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411700 2562 flags.go:64] FLAG: --log-json-split-stream="false" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411703 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411706 2562 flags.go:64] FLAG: --log-text-split-stream="false" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411708 2562 flags.go:64] FLAG: --logging-format="text" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411711 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411714 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411717 2562 flags.go:64] FLAG: --manifest-url="" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411720 2562 flags.go:64] FLAG: --manifest-url-header="" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411724 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411727 2562 flags.go:64] FLAG: --max-open-files="1000000" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411731 2562 flags.go:64] FLAG: --max-pods="110" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411734 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411739 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411742 2562 flags.go:64] FLAG: --memory-manager-policy="None" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411745 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411747 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411750 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411753 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411761 2562 flags.go:64] FLAG: --node-status-max-images="50" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411764 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411767 2562 flags.go:64] FLAG: --oom-score-adj="-999" Sep 09 20:50:23.419140 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411770 2562 flags.go:64] FLAG: --pod-cidr="" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411772 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411778 2562 flags.go:64] FLAG: --pod-manifest-path="" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411781 2562 flags.go:64] FLAG: --pod-max-pids="-1" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411784 2562 flags.go:64] FLAG: --pods-per-core="0" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411787 2562 flags.go:64] FLAG: --port="10250" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411790 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411793 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e1d3cdf0f49e1f3d" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411796 2562 flags.go:64] FLAG: --qos-reserved="" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411799 2562 flags.go:64] FLAG: --read-only-port="10255" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411802 2562 flags.go:64] FLAG: --register-node="true" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411805 2562 flags.go:64] FLAG: --register-with-taints="" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411808 2562 flags.go:64] FLAG: --registry-burst="10" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411811 2562 flags.go:64] FLAG: --registry-qps="5" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411814 2562 flags.go:64] FLAG: --reserved-cpus="" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411817 2562 flags.go:64] FLAG: --reserved-memory="" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411820 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411823 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411826 2562 flags.go:64] FLAG: --rotate-certificates="false" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411829 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411831 2562 flags.go:64] FLAG: --runonce="false" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411834 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411837 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411842 2562 flags.go:64] FLAG: --seccomp-default="false" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411845 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 09 20:50:23.419781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411848 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411851 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411854 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411857 2562 flags.go:64] FLAG: --storage-driver-password="root" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411859 2562 flags.go:64] FLAG: --storage-driver-secure="false" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411862 2562 flags.go:64] FLAG: --storage-driver-table="stats" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411865 2562 flags.go:64] FLAG: --storage-driver-user="root" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411867 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411870 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411873 2562 flags.go:64] FLAG: --system-cgroups="" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411876 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411881 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411884 2562 flags.go:64] FLAG: --tls-cert-file="" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411886 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411891 2562 flags.go:64] FLAG: --tls-min-version="" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411893 2562 flags.go:64] FLAG: --tls-private-key-file="" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411896 2562 flags.go:64] FLAG: --topology-manager-policy="none" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411899 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411902 2562 flags.go:64] FLAG: --topology-manager-scope="container" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411904 2562 flags.go:64] FLAG: --v="2" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411908 2562 flags.go:64] FLAG: --version="false" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411915 2562 flags.go:64] FLAG: --vmodule="" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411918 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.411921 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412043 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 20:50:23.420421 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412047 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412050 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412053 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412056 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412058 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412064 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412067 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412069 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412072 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412075 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412077 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412080 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412082 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412085 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412087 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412090 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412092 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412095 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412097 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412100 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 20:50:23.421033 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412102 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412106 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412110 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412113 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412116 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412118 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412122 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412125 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412128 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412132 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412134 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412137 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412139 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412142 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412145 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412147 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412150 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412155 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412157 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 20:50:23.421577 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412160 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412162 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412165 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412167 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412170 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412172 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412175 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412177 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412180 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412182 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412185 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412187 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412190 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412192 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412194 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412197 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412199 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412202 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412204 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412207 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 20:50:23.422052 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412209 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412212 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412214 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412218 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412221 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412223 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412226 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412228 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412230 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412255 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412262 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412265 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412267 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412270 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412272 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412275 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412277 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412280 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412282 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 20:50:23.422557 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412285 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412287 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412290 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412292 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412295 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412297 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412300 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412302 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412305 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412307 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412309 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412312 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412315 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412317 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412321 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412324 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412326 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412330 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412333 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 20:50:23.423029 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.412335 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 20:50:23.423515 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.413269 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 20:50:23.423515 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.420631 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 09 20:50:23.423515 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.420646 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 09 20:50:23.423515 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.421404 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 09 20:50:23.423515 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.421415 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 09 20:50:23.423515 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421476 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 20:50:23.423515 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421482 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 20:50:23.423515 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421485 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 20:50:23.423515 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421488 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 20:50:23.423515 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421491 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 20:50:23.423515 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421494 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 20:50:23.423515 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421497 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 20:50:23.423515 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421500 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 20:50:23.423515 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421502 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 20:50:23.423515 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421506 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421508 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421511 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421513 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421516 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421518 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421521 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421523 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421526 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421529 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421533 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421536 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421538 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421541 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421545 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421549 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421553 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421556 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421559 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 20:50:23.424014 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421561 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421564 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421567 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421570 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421572 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421575 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421578 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421581 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421583 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421586 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421588 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421591 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421593 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421596 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421598 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421601 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421603 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421605 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421609 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421611 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 20:50:23.424512 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421614 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421617 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421620 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421623 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421625 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421628 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421630 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421633 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421635 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421639 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421642 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421644 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421646 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421649 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421652 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421655 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421657 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421660 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421662 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 20:50:23.425035 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421666 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421669 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421672 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421675 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421677 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421680 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421682 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421685 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421687 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421690 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421692 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421695 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421697 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421700 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421702 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421704 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421707 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421709 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421711 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421714 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 20:50:23.425514 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421716 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 20:50:23.426002 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421719 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 20:50:23.426002 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421722 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 20:50:23.426002 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421725 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 20:50:23.426002 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421728 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 20:50:23.426002 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421730 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 20:50:23.426002 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421733 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 20:50:23.426002 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421736 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 20:50:23.426002 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421738 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 20:50:23.426002 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421741 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 20:50:23.426002 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421743 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 20:50:23.426002 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421746 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 20:50:23.426002 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.421750 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 20:50:23.426002 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421888 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 20:50:23.426002 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421893 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 20:50:23.426002 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421896 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421899 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421902 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421905 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421908 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421910 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421913 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421917 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421921 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421924 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421927 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421930 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421933 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421935 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421938 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421941 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421943 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421946 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421949 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421951 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 20:50:23.426476 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421955 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421958 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421960 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421963 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421966 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421968 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421970 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421974 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421977 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421980 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421983 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421985 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421988 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421991 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421994 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421997 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.421999 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422002 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422005 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 20:50:23.426953 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422008 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422011 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422013 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422015 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422018 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422020 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422023 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422026 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422028 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422031 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422033 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422036 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422038 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422041 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422044 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422047 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422049 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422052 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422056 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422059 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 20:50:23.427429 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422062 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422065 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422067 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422070 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422072 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422075 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422077 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422080 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422082 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422085 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422087 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422090 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422093 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422095 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422098 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422100 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422102 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422105 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422107 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422110 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 20:50:23.427908 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422112 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422115 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422117 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422120 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422122 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422125 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422127 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422130 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422133 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422135 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422138 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422140 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422143 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422145 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422148 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422150 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422153 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 20:50:23.428422 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:23.422155 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 20:50:23.428872 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.422160 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 20:50:23.428872 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.422279 2562 server.go:962] "Client rotation is on, will bootstrap in background" Sep 09 20:50:23.428872 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.424337 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 09 20:50:23.428872 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.425153 2562 server.go:1019] "Starting client certificate rotation" Sep 09 20:50:23.428872 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.425263 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 20:50:23.428872 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.425876 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 20:50:23.428872 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.427448 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 20:50:23.428872 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.428622 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 20:50:23.440175 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.440142 2562 log.go:25] "Validated CRI v1 runtime API" Sep 09 20:50:23.445066 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.445046 2562 log.go:25] "Validated CRI v1 image API" Sep 09 20:50:23.446629 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.446598 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 09 20:50:23.449710 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.449690 2562 fs.go:135] Filesystem UUIDs: map[15265d0a-abe0-4c7f-b11d-49875d74447c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7fb97e0e-8fa9-4ea7-889c-6bdfa64325e7:/dev/nvme0n1p4] Sep 09 20:50:23.449785 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.449711 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 09 20:50:23.454899 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.454793 2562 manager.go:216] Machine: {Timestamp:2026-09-09 20:50:23.453446719 +0000 UTC m=+0.292645841 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3131478 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec236775af9277c1ffef987d47f66148 SystemUUID:ec236775-af92-77c1-ffef-987d47f66148 BootID:1b249e23-9e4d-4254-adaa-eb856999ae58 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6f:be:4c:26:5f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6f:be:4c:26:5f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:e3:e8:0a:38:1d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 09 20:50:23.454899 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.454896 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 09 20:50:23.455022 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.454977 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 09 20:50:23.458124 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.458097 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 09 20:50:23.458282 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.458126 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-83.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 09 20:50:23.458866 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.458857 2562 topology_manager.go:138] "Creating topology manager with none policy" Sep 09 20:50:23.458897 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.458869 2562 container_manager_linux.go:309] "Creating device plugin manager" Sep 09 20:50:23.458897 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.458877 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 20:50:23.460464 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.460451 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 20:50:23.462142 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.462130 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 09 20:50:23.466697 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.466684 2562 state_mem.go:36] "Initialized new in-memory state store" Sep 09 20:50:23.466828 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.466819 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 09 20:50:23.466866 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.466857 2562 kubelet.go:486] "Attempting to sync node with API server" Sep 09 20:50:23.466906 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.466874 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 09 20:50:23.466906 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.466905 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 09 20:50:23.466964 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.466918 2562 kubelet.go:398] "Adding apiserver pod source" Sep 09 20:50:23.466964 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.466941 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 09 20:50:23.467293 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.467283 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 20:50:23.467334 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.467302 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 20:50:23.469622 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.469603 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 09 20:50:23.472492 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.472479 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 09 20:50:23.472554 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.472510 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 09 20:50:23.472686 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.472667 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 09 20:50:23.472776 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.472694 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 09 20:50:23.472776 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.472704 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 09 20:50:23.472776 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.472713 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 09 20:50:23.472776 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.472727 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 09 20:50:23.472776 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.472740 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 09 20:50:23.473307 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.473296 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 09 20:50:23.473307 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.473307 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 09 20:50:23.473411 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.473316 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 09 20:50:23.473411 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.473321 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 09 20:50:23.473411 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.473330 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 09 20:50:23.473411 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.473341 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 09 20:50:23.474201 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.474191 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 09 20:50:23.474201 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.474201 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 09 20:50:23.477213 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.477193 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 20:50:23.478428 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.478412 2562 server.go:1268] "Started kubelet" Sep 09 20:50:23.478564 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.478532 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 09 20:50:23.478796 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.478702 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 09 20:50:23.478875 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.478821 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 09 20:50:23.479312 ip-10-0-139-83 systemd[1]: Started Kubernetes Kubelet. Sep 09 20:50:23.480516 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.480398 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 09 20:50:23.480721 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.480704 2562 server.go:310] "Adding debug handlers to kubelet server" Sep 09 20:50:23.482717 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.482689 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 09 20:50:23.484815 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.484800 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-83.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 09 20:50:23.485572 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.485552 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-83.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 09 20:50:23.485635 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.485550 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 09 20:50:23.486421 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.485306 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-83.ec2.internal.18d3c2382a3640c9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-83.ec2.internal,UID:ip-10-0-139-83.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-83.ec2.internal,},FirstTimestamp:2026-09-09 20:50:23.478374601 +0000 UTC m=+0.317573724,LastTimestamp:2026-09-09 20:50:23.478374601 +0000 UTC m=+0.317573724,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-83.ec2.internal,}" Sep 09 20:50:23.486880 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.486868 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 09 20:50:23.488216 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.488199 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 09 20:50:23.488469 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.488313 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-83.ec2.internal\" not found" Sep 09 20:50:23.488980 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.488952 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 09 20:50:23.488980 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.488971 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 09 20:50:23.488980 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.488952 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 09 20:50:23.489154 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.489042 2562 factory.go:55] Registering systemd factory Sep 09 20:50:23.489154 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.489054 2562 factory.go:223] Registration of the systemd container factory successfully Sep 09 20:50:23.489154 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.489060 2562 reconstruct.go:97] "Volume reconstruction finished" Sep 09 20:50:23.489154 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.489066 2562 reconciler.go:29] "Reconciler: start to sync state" Sep 09 20:50:23.489388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.489286 2562 factory.go:153] Registering CRI-O factory Sep 09 20:50:23.489388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.489296 2562 factory.go:223] Registration of the crio container factory successfully Sep 09 20:50:23.489388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.489358 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 09 20:50:23.489388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.489375 2562 factory.go:103] Registering Raw factory Sep 09 20:50:23.489388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.489389 2562 manager.go:1196] Started watching for new ooms in manager Sep 09 20:50:23.490149 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.490130 2562 manager.go:349] Starting recovery of all containers Sep 09 20:50:23.497891 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.497854 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 09 20:50:23.502501 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.502472 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 09 20:50:23.502614 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.502488 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-83.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 09 20:50:23.503446 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.503430 2562 manager.go:354] Recovery completed Sep 09 20:50:23.507609 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.507597 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 20:50:23.509400 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.509381 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasSufficientMemory" Sep 09 20:50:23.509499 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.509410 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 20:50:23.509499 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.509421 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasSufficientPID" Sep 09 20:50:23.509882 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.509867 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 09 20:50:23.509882 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.509880 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 09 20:50:23.509990 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.509886 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 20:50:23.509990 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.509895 2562 state_mem.go:36] "Initialized new in-memory state store" Sep 09 20:50:23.510550 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.510535 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasSufficientMemory" Sep 09 20:50:23.510613 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.510561 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 20:50:23.510613 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.510571 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasSufficientPID" Sep 09 20:50:23.512143 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.512131 2562 policy_none.go:49] "None policy: Start" Sep 09 20:50:23.512189 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.512147 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 09 20:50:23.512189 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.512156 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 09 20:50:23.512336 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.512266 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-83.ec2.internal.18d3c2382c0fa22b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-83.ec2.internal,UID:ip-10-0-139-83.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-83.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-83.ec2.internal,},FirstTimestamp:2026-09-09 20:50:23.509398059 +0000 UTC m=+0.348597180,LastTimestamp:2026-09-09 20:50:23.509398059 +0000 UTC m=+0.348597180,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-83.ec2.internal,}" Sep 09 20:50:23.514217 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.514205 2562 policy_none.go:47] "Start" Sep 09 20:50:23.523098 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.523024 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-83.ec2.internal.18d3c2382c0fe48c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-83.ec2.internal,UID:ip-10-0-139-83.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-83.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-83.ec2.internal,},FirstTimestamp:2026-09-09 20:50:23.509415052 +0000 UTC m=+0.348614173,LastTimestamp:2026-09-09 20:50:23.509415052 +0000 UTC m=+0.348614173,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-83.ec2.internal,}" Sep 09 20:50:23.534281 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.534182 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-83.ec2.internal.18d3c2382c1009d4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-83.ec2.internal,UID:ip-10-0-139-83.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-83.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-83.ec2.internal,},FirstTimestamp:2026-09-09 20:50:23.509424596 +0000 UTC m=+0.348623716,LastTimestamp:2026-09-09 20:50:23.509424596 +0000 UTC m=+0.348623716,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-83.ec2.internal,}" Sep 09 20:50:23.540207 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.540186 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zrgtx" Sep 09 20:50:23.543385 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.543314 2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-139-83.ec2.internal.18d3c2382c0fa22b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-83.ec2.internal.18d3c2382c0fa22b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-83.ec2.internal,UID:ip-10-0-139-83.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-83.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-83.ec2.internal,},FirstTimestamp:2026-09-09 20:50:23.509398059 +0000 UTC m=+0.348597180,LastTimestamp:2026-09-09 20:50:23.510549909 +0000 UTC m=+0.349749030,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-83.ec2.internal,}" Sep 09 20:50:23.570842 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.550520 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zrgtx" Sep 09 20:50:23.570842 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.550801 2562 manager.go:337] "Starting Device Plugin manager" Sep 09 20:50:23.570842 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.550843 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 09 20:50:23.570842 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.550853 2562 server.go:87] "Starting device plugin registration server" Sep 09 20:50:23.570842 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.551261 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 09 20:50:23.570842 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.551277 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 09 20:50:23.570842 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.551395 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 09 20:50:23.570842 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.551482 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 09 20:50:23.570842 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.551491 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 09 20:50:23.570842 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.551951 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 09 20:50:23.570842 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.552005 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-83.ec2.internal\" not found" Sep 09 20:50:23.570842 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.554582 2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-139-83.ec2.internal.18d3c2382c0fe48c\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-83.ec2.internal.18d3c2382c0fe48c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-83.ec2.internal,UID:ip-10-0-139-83.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-83.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-83.ec2.internal,},FirstTimestamp:2026-09-09 20:50:23.509415052 +0000 UTC m=+0.348614173,LastTimestamp:2026-09-09 20:50:23.510565645 +0000 UTC m=+0.349764765,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-83.ec2.internal,}" Sep 09 20:50:23.632204 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.632168 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 09 20:50:23.632353 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.632227 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 09 20:50:23.632353 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.632260 2562 kubelet.go:2448] "Starting kubelet main sync loop" Sep 09 20:50:23.632353 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.632296 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 09 20:50:23.634661 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.634636 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 20:50:23.651586 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.651568 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 20:50:23.652437 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.652375 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasSufficientMemory" Sep 09 20:50:23.652437 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.652408 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 20:50:23.652437 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.652417 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasSufficientPID" Sep 09 20:50:23.652437 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.652439 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-83.ec2.internal" Sep 09 20:50:23.660778 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.660758 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-83.ec2.internal" Sep 09 20:50:23.660827 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.660783 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-83.ec2.internal\": node \"ip-10-0-139-83.ec2.internal\" not found" Sep 09 20:50:23.677930 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.677907 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-83.ec2.internal\" not found" Sep 09 20:50:23.732820 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.732793 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-83.ec2.internal"] Sep 09 20:50:23.732926 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.732864 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 20:50:23.733786 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.733771 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasSufficientMemory" Sep 09 20:50:23.733849 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.733809 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 20:50:23.733849 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.733819 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasSufficientPID" Sep 09 20:50:23.736180 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.736166 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 20:50:23.736323 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.736298 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal" Sep 09 20:50:23.736356 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.736330 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 20:50:23.736893 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.736876 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasSufficientMemory" Sep 09 20:50:23.736949 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.736907 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 20:50:23.736949 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.736917 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasSufficientPID" Sep 09 20:50:23.736949 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.736881 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasSufficientMemory" Sep 09 20:50:23.737043 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.736965 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 20:50:23.737043 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.736979 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasSufficientPID" Sep 09 20:50:23.739224 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.739212 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-83.ec2.internal" Sep 09 20:50:23.739287 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.739254 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 20:50:23.739901 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.739881 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasSufficientMemory" Sep 09 20:50:23.740008 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.739917 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 20:50:23.740008 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.739944 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasSufficientPID" Sep 09 20:50:23.764046 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.764023 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-83.ec2.internal\" not found" node="ip-10-0-139-83.ec2.internal" Sep 09 20:50:23.770133 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.770090 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-83.ec2.internal\" not found" node="ip-10-0-139-83.ec2.internal" Sep 09 20:50:23.778184 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.778162 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-83.ec2.internal\" not found" Sep 09 20:50:23.790794 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.790770 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c92fd857038978a287ea82826bec9300-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal\" (UID: \"c92fd857038978a287ea82826bec9300\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal" Sep 09 20:50:23.790867 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.790797 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c92fd857038978a287ea82826bec9300-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal\" (UID: \"c92fd857038978a287ea82826bec9300\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal" Sep 09 20:50:23.790867 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.790815 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2df482739528dd8ed1304969d8f65a5c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-83.ec2.internal\" (UID: \"2df482739528dd8ed1304969d8f65a5c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-83.ec2.internal" Sep 09 20:50:23.878636 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.878598 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-83.ec2.internal\" not found" Sep 09 20:50:23.891028 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.890999 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c92fd857038978a287ea82826bec9300-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal\" (UID: \"c92fd857038978a287ea82826bec9300\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal" Sep 09 20:50:23.891130 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.891024 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c92fd857038978a287ea82826bec9300-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal\" (UID: \"c92fd857038978a287ea82826bec9300\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal" Sep 09 20:50:23.891130 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.891086 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c92fd857038978a287ea82826bec9300-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal\" (UID: \"c92fd857038978a287ea82826bec9300\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal" Sep 09 20:50:23.891130 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.891114 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2df482739528dd8ed1304969d8f65a5c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-83.ec2.internal\" (UID: \"2df482739528dd8ed1304969d8f65a5c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-83.ec2.internal" Sep 09 20:50:23.891275 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.891162 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2df482739528dd8ed1304969d8f65a5c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-83.ec2.internal\" (UID: \"2df482739528dd8ed1304969d8f65a5c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-83.ec2.internal" Sep 09 20:50:23.891275 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:23.891173 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c92fd857038978a287ea82826bec9300-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal\" (UID: \"c92fd857038978a287ea82826bec9300\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal" Sep 09 20:50:23.979468 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:23.979406 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-83.ec2.internal\" not found" Sep 09 20:50:24.066998 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.066965 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal" Sep 09 20:50:24.073150 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.073130 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-83.ec2.internal" Sep 09 20:50:24.080054 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:24.080031 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-83.ec2.internal\" not found" Sep 09 20:50:24.180515 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:24.180476 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-83.ec2.internal\" not found" Sep 09 20:50:24.281092 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:24.281018 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-83.ec2.internal\" not found" Sep 09 20:50:24.381664 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:24.381631 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-83.ec2.internal\" not found" Sep 09 20:50:24.425206 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.425164 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 09 20:50:24.425659 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.425338 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 20:50:24.482737 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:24.482704 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-83.ec2.internal\" not found" Sep 09 20:50:24.502461 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.502439 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 20:50:24.539600 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.539548 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 20:50:24.553547 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.553520 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 20:45:23 +0000 UTC" deadline="2028-04-01 05:23:03.548802286 +0000 UTC" Sep 09 20:50:24.553547 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.553545 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13664h32m38.995260169s" Sep 09 20:50:24.583690 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:24.583663 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-83.ec2.internal\" not found" Sep 09 20:50:24.592030 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.592015 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 20:50:24.593496 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.593479 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasSufficientMemory" Sep 09 20:50:24.593589 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.593508 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 20:50:24.593589 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.593518 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeHasSufficientPID" Sep 09 20:50:24.630102 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.630080 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 20:50:24.689323 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.689293 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal" Sep 09 20:50:24.700433 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.700412 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 20:50:24.701452 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.701438 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-83.ec2.internal" Sep 09 20:50:24.710413 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.710397 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 20:50:24.782594 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:24.782564 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2df482739528dd8ed1304969d8f65a5c.slice/crio-7325a0d069dc6f9ffa0556bdd42804504305a9d7e5e7e9d93affe9a382b8b044 WatchSource:0}: Error finding container 7325a0d069dc6f9ffa0556bdd42804504305a9d7e5e7e9d93affe9a382b8b044: Status 404 returned error can't find the container with id 7325a0d069dc6f9ffa0556bdd42804504305a9d7e5e7e9d93affe9a382b8b044 Sep 09 20:50:24.782804 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:24.782791 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc92fd857038978a287ea82826bec9300.slice/crio-0bff4fcddb9de39a093db06db8fd50bf6d5a35863481502c2b3f240172d39e1e WatchSource:0}: Error finding container 0bff4fcddb9de39a093db06db8fd50bf6d5a35863481502c2b3f240172d39e1e: Status 404 returned error can't find the container with id 0bff4fcddb9de39a093db06db8fd50bf6d5a35863481502c2b3f240172d39e1e Sep 09 20:50:24.788850 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:24.788835 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 20:50:25.468532 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.468451 2562 apiserver.go:52] "Watching apiserver" Sep 09 20:50:25.475682 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.475657 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 09 20:50:25.477481 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.477455 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-x59k7","kube-system/konnectivity-agent-mgrlj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78","openshift-multus/multus-krlkq","openshift-multus/network-metrics-daemon-lqd29","openshift-network-diagnostics/network-check-target-vtb42","openshift-network-operator/iptables-alerter-4h5r5","kube-system/kube-apiserver-proxy-ip-10-0-139-83.ec2.internal","openshift-cluster-node-tuning-operator/tuned-jkgsv","openshift-dns/node-resolver-mvftd","openshift-image-registry/node-ca-sgvv4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal","openshift-multus/multus-additional-cni-plugins-prbnq","openshift-ovn-kubernetes/ovnkube-node-kjz44"] Sep 09 20:50:25.480065 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.480045 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4h5r5" Sep 09 20:50:25.481868 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.481843 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.482919 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.482897 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bnn6j\"" Sep 09 20:50:25.483163 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.483141 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 09 20:50:25.483390 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.483339 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 09 20:50:25.483524 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.483486 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 09 20:50:25.485601 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.484507 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 09 20:50:25.485601 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.484625 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 09 20:50:25.485601 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.484690 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 09 20:50:25.485601 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.484861 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 09 20:50:25.485601 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.484902 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pp5xn\"" Sep 09 20:50:25.485601 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.485085 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 09 20:50:25.485601 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.485170 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.486456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.486083 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.487104 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.486956 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 09 20:50:25.488737 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.487964 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 09 20:50:25.488737 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.488503 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q7nkk\"" Sep 09 20:50:25.488737 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.488508 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 09 20:50:25.488737 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.488598 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-g5r2f\"" Sep 09 20:50:25.488737 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.488651 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 09 20:50:25.490102 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.489048 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 09 20:50:25.490102 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.489815 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:25.490102 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:25.489884 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:25.492775 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.492641 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:25.492863 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:25.492778 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:25.495074 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.493885 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.496462 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.496340 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:25.496462 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:25.496421 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:25.497485 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.497328 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cjztd\"" Sep 09 20:50:25.497566 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.497494 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 09 20:50:25.497657 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.497637 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 09 20:50:25.498559 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498543 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-etc-kubernetes\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.498644 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498571 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8d88\" (UniqueName: \"kubernetes.io/projected/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-kube-api-access-n8d88\") pod \"network-metrics-daemon-lqd29\" (UID: \"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5\") " pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:25.498644 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498595 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.498644 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498623 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-var-lib-cni-multus\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.498885 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498645 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/46ffa1a9-4a22-4274-b366-30bb80a2fa85-multus-daemon-config\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.498885 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498665 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkbxz\" (UniqueName: \"kubernetes.io/projected/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-kube-api-access-pkbxz\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.498885 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498695 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-cnibin\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.498885 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498715 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-cni-binary-copy\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.498885 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498736 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-var-lib-cni-bin\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.498885 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498755 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-hostroot\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.498885 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498778 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.498885 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498798 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-socket-dir\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.498885 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498828 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-multus-socket-dir-parent\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.498885 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498852 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7-host-slash\") pod \"iptables-alerter-4h5r5\" (UID: \"9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7\") " pod="openshift-network-operator/iptables-alerter-4h5r5" Sep 09 20:50:25.498885 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498874 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-system-cni-dir\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.499313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498897 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-cnibin\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.499313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498918 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-run-k8s-cni-cncf-io\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.499313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498941 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.499313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498963 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flvj7\" (UniqueName: \"kubernetes.io/projected/5f7991ef-372f-4d93-b7b8-25c7bad0528d-kube-api-access-flvj7\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.499313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.498984 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-registration-dir\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.499313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.499005 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-etc-selinux\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.499313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.499026 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7fb5\" (UniqueName: \"kubernetes.io/projected/46ffa1a9-4a22-4274-b366-30bb80a2fa85-kube-api-access-g7fb5\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.499313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.499051 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-sys-fs\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.499313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.499070 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/46ffa1a9-4a22-4274-b366-30bb80a2fa85-cni-binary-copy\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.499313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.499090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-multus-conf-dir\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.499313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.499110 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs\") pod \"network-metrics-daemon-lqd29\" (UID: \"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5\") " pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:25.499313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.499131 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfwd7\" (UniqueName: \"kubernetes.io/projected/9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7-kube-api-access-dfwd7\") pod \"iptables-alerter-4h5r5\" (UID: \"9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7\") " pod="openshift-network-operator/iptables-alerter-4h5r5" Sep 09 20:50:25.499313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.499161 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-system-cni-dir\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.499313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.499185 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-device-dir\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.499313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.499206 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7-iptables-alerter-script\") pod \"iptables-alerter-4h5r5\" (UID: \"9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7\") " pod="openshift-network-operator/iptables-alerter-4h5r5" Sep 09 20:50:25.499313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.499251 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.500064 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.499272 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-multus-cni-dir\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.500064 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.499293 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-run-netns\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.500064 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.499314 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-var-lib-kubelet\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.500064 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.499341 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-run-multus-certs\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.500064 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.499824 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mgrlj" Sep 09 20:50:25.504402 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.501753 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 09 20:50:25.504402 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.502133 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-smpv9\"" Sep 09 20:50:25.504402 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.502200 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 09 20:50:25.504402 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.503847 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 20:50:25.506427 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.506311 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mvftd" Sep 09 20:50:25.508836 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.508419 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 09 20:50:25.508836 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.508644 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 09 20:50:25.509084 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.509070 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sgvv4" Sep 09 20:50:25.509777 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.509764 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7dp22\"" Sep 09 20:50:25.510512 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.510495 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.511774 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.511394 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 09 20:50:25.511774 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.511430 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9rnwg\"" Sep 09 20:50:25.511774 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.511523 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 09 20:50:25.511774 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.511591 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 09 20:50:25.512531 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.512508 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 09 20:50:25.512669 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.512656 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 09 20:50:25.512859 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.512843 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rz22j\"" Sep 09 20:50:25.513555 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.513359 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 09 20:50:25.513555 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.513430 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 09 20:50:25.513555 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.513429 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 09 20:50:25.513731 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.513604 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 09 20:50:25.519181 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.519080 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9z8tr" Sep 09 20:50:25.526062 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.525820 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9z8tr" Sep 09 20:50:25.590901 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.590873 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 09 20:50:25.600090 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600066 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-modprobe-d\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.600215 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600100 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/79c3cb22-0efb-42bc-8697-ad6431a2a874-dbus\") pod \"global-pull-secret-syncer-x59k7\" (UID: \"79c3cb22-0efb-42bc-8697-ad6431a2a874\") " pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:25.600215 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600121 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-etc-openvswitch\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.600215 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600141 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-log-socket\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.600215 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600168 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-system-cni-dir\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.600215 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600191 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.600412 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600213 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjsvx\" (UniqueName: \"kubernetes.io/projected/d4d297c3-08db-42da-8621-b9b3d6ff8b9f-kube-api-access-wjsvx\") pod \"node-resolver-mvftd\" (UID: \"d4d297c3-08db-42da-8621-b9b3d6ff8b9f\") " pod="openshift-dns/node-resolver-mvftd" Sep 09 20:50:25.600412 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600259 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-var-lib-openvswitch\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.600412 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600281 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/35a33f42-b524-465b-bb4a-ab3526a6a6d9-ovn-node-metrics-cert\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.600412 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600304 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-multus-cni-dir\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.600412 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600344 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-run-netns\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.600412 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600365 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-var-lib-kubelet\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.600412 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600387 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-run-multus-certs\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.600412 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600411 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-etc-kubernetes\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.600658 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600440 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.600658 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600461 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-run-netns\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.600658 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600485 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-run-ovn\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.600658 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600506 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-var-lib-cni-multus\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.600658 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600527 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-slash\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.600658 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600547 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-cni-bin\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.600658 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600567 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-run\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.600658 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600587 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-cni-binary-copy\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.600658 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600610 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-run-openvswitch\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.600658 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600632 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/35a33f42-b524-465b-bb4a-ab3526a6a6d9-ovnkube-script-lib\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.600658 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600654 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-tuned\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600676 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-var-lib-cni-bin\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600692 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600704 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-socket-dir\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600726 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-multus-socket-dir-parent\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600746 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-system-cni-dir\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbgnn\" (UniqueName: \"kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn\") pod \"network-check-target-vtb42\" (UID: \"f57dac14-ec36-409f-b8ef-e7a993c313ae\") " pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600795 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93-host\") pod \"node-ca-sgvv4\" (UID: \"51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93\") " pod="openshift-image-registry/node-ca-sgvv4" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600807 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/35a33f42-b524-465b-bb4a-ab3526a6a6d9-env-overrides\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600821 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-cnibin\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600832 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4d297c3-08db-42da-8621-b9b3d6ff8b9f-tmp-dir\") pod \"node-resolver-mvftd\" (UID: \"d4d297c3-08db-42da-8621-b9b3d6ff8b9f\") " pod="openshift-dns/node-resolver-mvftd" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600845 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmj8v\" (UniqueName: \"kubernetes.io/projected/51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93-kube-api-access-kmj8v\") pod \"node-ca-sgvv4\" (UID: \"51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93\") " pod="openshift-image-registry/node-ca-sgvv4" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600858 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-sysctl-conf\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600872 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-lib-modules\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600885 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-host\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600897 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-tmp\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600914 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret\") pod \"global-pull-secret-syncer-x59k7\" (UID: \"79c3cb22-0efb-42bc-8697-ad6431a2a874\") " pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:25.601078 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600928 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-etc-selinux\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600941 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7fb5\" (UniqueName: \"kubernetes.io/projected/46ffa1a9-4a22-4274-b366-30bb80a2fa85-kube-api-access-g7fb5\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600953 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93-serviceca\") pod \"node-ca-sgvv4\" (UID: \"51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93\") " pod="openshift-image-registry/node-ca-sgvv4" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600967 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/46ffa1a9-4a22-4274-b366-30bb80a2fa85-cni-binary-copy\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.600982 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfwd7\" (UniqueName: \"kubernetes.io/projected/9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7-kube-api-access-dfwd7\") pod \"iptables-alerter-4h5r5\" (UID: \"9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7\") " pod="openshift-network-operator/iptables-alerter-4h5r5" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601014 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-device-dir\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601029 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7-iptables-alerter-script\") pod \"iptables-alerter-4h5r5\" (UID: \"9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7\") " pod="openshift-network-operator/iptables-alerter-4h5r5" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601043 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601069 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-sysconfig\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601081 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-kubernetes\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601094 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-systemd\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601108 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8d88\" (UniqueName: \"kubernetes.io/projected/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-kube-api-access-n8d88\") pod \"network-metrics-daemon-lqd29\" (UID: \"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5\") " pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601121 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-sys\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601135 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/46ffa1a9-4a22-4274-b366-30bb80a2fa85-multus-daemon-config\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601157 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkbxz\" (UniqueName: \"kubernetes.io/projected/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-kube-api-access-pkbxz\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-node-log\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601183 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-sysctl-d\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.601704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601195 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-var-lib-kubelet\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.602456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601208 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-cnibin\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.602456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601220 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-systemd-units\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.602456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601256 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-run-systemd\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.602456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601277 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/79c3cb22-0efb-42bc-8697-ad6431a2a874-kubelet-config\") pod \"global-pull-secret-syncer-x59k7\" (UID: \"79c3cb22-0efb-42bc-8697-ad6431a2a874\") " pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:25.602456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601301 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5e788c00-eca3-4b87-98e5-89bd93f0abd0-agent-certs\") pod \"konnectivity-agent-mgrlj\" (UID: \"5e788c00-eca3-4b87-98e5-89bd93f0abd0\") " pod="kube-system/konnectivity-agent-mgrlj" Sep 09 20:50:25.602456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601315 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5e788c00-eca3-4b87-98e5-89bd93f0abd0-konnectivity-ca\") pod \"konnectivity-agent-mgrlj\" (UID: \"5e788c00-eca3-4b87-98e5-89bd93f0abd0\") " pod="kube-system/konnectivity-agent-mgrlj" Sep 09 20:50:25.602456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601332 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-hostroot\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.602456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601357 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7-host-slash\") pod \"iptables-alerter-4h5r5\" (UID: \"9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7\") " pod="openshift-network-operator/iptables-alerter-4h5r5" Sep 09 20:50:25.602456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601372 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d4d297c3-08db-42da-8621-b9b3d6ff8b9f-hosts-file\") pod \"node-resolver-mvftd\" (UID: \"d4d297c3-08db-42da-8621-b9b3d6ff8b9f\") " pod="openshift-dns/node-resolver-mvftd" Sep 09 20:50:25.602456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601388 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpkg4\" (UniqueName: \"kubernetes.io/projected/35a33f42-b524-465b-bb4a-ab3526a6a6d9-kube-api-access-jpkg4\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.602456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601404 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnqsd\" (UniqueName: \"kubernetes.io/projected/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-kube-api-access-wnqsd\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.602456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601418 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-run-k8s-cni-cncf-io\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.602456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601434 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.602456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601448 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-cni-netd\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.602456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601462 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flvj7\" (UniqueName: \"kubernetes.io/projected/5f7991ef-372f-4d93-b7b8-25c7bad0528d-kube-api-access-flvj7\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.602456 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601479 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-registration-dir\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601493 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-kubelet\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601508 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/35a33f42-b524-465b-bb4a-ab3526a6a6d9-ovnkube-config\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-sys-fs\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601557 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-multus-conf-dir\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601570 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs\") pod \"network-metrics-daemon-lqd29\" (UID: \"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5\") " pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601584 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.601723 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-system-cni-dir\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.602006 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-multus-cni-dir\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.602057 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-run-netns\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.602097 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-var-lib-kubelet\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.602123 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-var-lib-cni-multus\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.602151 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-run-multus-certs\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.602278 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-cnibin\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.602311 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-var-lib-cni-bin\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.602342 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-etc-kubernetes\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.602381 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.602520 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.603176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.602614 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-socket-dir\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.602661 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-multus-socket-dir-parent\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.602696 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-system-cni-dir\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.602769 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-etc-selinux\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.602879 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-device-dir\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.603027 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-cni-binary-copy\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.603303 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-multus-conf-dir\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:25.603380 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.603379 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.603424 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/46ffa1a9-4a22-4274-b366-30bb80a2fa85-cni-binary-copy\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.603445 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-host-run-k8s-cni-cncf-io\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:25.603452 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs podName:ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5 nodeName:}" failed. No retries permitted until 2026-09-09 20:50:26.103412978 +0000 UTC m=+2.942612102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs") pod "network-metrics-daemon-lqd29" (UID: "ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.603482 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/46ffa1a9-4a22-4274-b366-30bb80a2fa85-hostroot\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.603487 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7-host-slash\") pod \"iptables-alerter-4h5r5\" (UID: \"9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7\") " pod="openshift-network-operator/iptables-alerter-4h5r5" Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.603533 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-sys-fs\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.603564 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5f7991ef-372f-4d93-b7b8-25c7bad0528d-registration-dir\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.603584 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/46ffa1a9-4a22-4274-b366-30bb80a2fa85-multus-daemon-config\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.603979 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.603628 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-cnibin\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.604607 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.603963 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.604607 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.604020 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7-iptables-alerter-script\") pod \"iptables-alerter-4h5r5\" (UID: \"9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7\") " pod="openshift-network-operator/iptables-alerter-4h5r5" Sep 09 20:50:25.608451 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.608425 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 09 20:50:25.616305 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.616096 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfwd7\" (UniqueName: \"kubernetes.io/projected/9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7-kube-api-access-dfwd7\") pod \"iptables-alerter-4h5r5\" (UID: \"9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7\") " pod="openshift-network-operator/iptables-alerter-4h5r5" Sep 09 20:50:25.616426 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.616107 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8d88\" (UniqueName: \"kubernetes.io/projected/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-kube-api-access-n8d88\") pod \"network-metrics-daemon-lqd29\" (UID: \"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5\") " pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:25.616426 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.616117 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flvj7\" (UniqueName: \"kubernetes.io/projected/5f7991ef-372f-4d93-b7b8-25c7bad0528d-kube-api-access-flvj7\") pod \"aws-ebs-csi-driver-node-rss78\" (UID: \"5f7991ef-372f-4d93-b7b8-25c7bad0528d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.616426 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.616422 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7fb5\" (UniqueName: \"kubernetes.io/projected/46ffa1a9-4a22-4274-b366-30bb80a2fa85-kube-api-access-g7fb5\") pod \"multus-krlkq\" (UID: \"46ffa1a9-4a22-4274-b366-30bb80a2fa85\") " pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.616608 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.616576 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkbxz\" (UniqueName: \"kubernetes.io/projected/e26cba9c-5ba0-44c8-9d41-c63dad722bbe-kube-api-access-pkbxz\") pod \"multus-additional-cni-plugins-prbnq\" (UID: \"e26cba9c-5ba0-44c8-9d41-c63dad722bbe\") " pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.637673 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.637636 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-83.ec2.internal" event={"ID":"2df482739528dd8ed1304969d8f65a5c","Type":"ContainerStarted","Data":"7325a0d069dc6f9ffa0556bdd42804504305a9d7e5e7e9d93affe9a382b8b044"} Sep 09 20:50:25.639704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.639664 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal" event={"ID":"c92fd857038978a287ea82826bec9300","Type":"ContainerStarted","Data":"0bff4fcddb9de39a093db06db8fd50bf6d5a35863481502c2b3f240172d39e1e"} Sep 09 20:50:25.702749 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702721 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93-serviceca\") pod \"node-ca-sgvv4\" (UID: \"51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93\") " pod="openshift-image-registry/node-ca-sgvv4" Sep 09 20:50:25.702895 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702761 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.702895 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702780 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-sysconfig\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.702895 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702799 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-kubernetes\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.702895 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702819 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-systemd\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.702895 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702832 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.702895 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702839 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-sys\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.702895 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702879 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-node-log\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.702895 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702888 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-systemd\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.702895 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702885 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-kubernetes\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702901 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-sysctl-d\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702923 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-sysconfig\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702935 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-sys\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702925 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-var-lib-kubelet\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702959 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-var-lib-kubelet\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702967 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-node-log\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702973 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-systemd-units\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.702997 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-run-systemd\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703000 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-systemd-units\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703019 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/79c3cb22-0efb-42bc-8697-ad6431a2a874-kubelet-config\") pod \"global-pull-secret-syncer-x59k7\" (UID: \"79c3cb22-0efb-42bc-8697-ad6431a2a874\") " pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703040 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5e788c00-eca3-4b87-98e5-89bd93f0abd0-agent-certs\") pod \"konnectivity-agent-mgrlj\" (UID: \"5e788c00-eca3-4b87-98e5-89bd93f0abd0\") " pod="kube-system/konnectivity-agent-mgrlj" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703052 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-run-systemd\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5e788c00-eca3-4b87-98e5-89bd93f0abd0-konnectivity-ca\") pod \"konnectivity-agent-mgrlj\" (UID: \"5e788c00-eca3-4b87-98e5-89bd93f0abd0\") " pod="kube-system/konnectivity-agent-mgrlj" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703088 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d4d297c3-08db-42da-8621-b9b3d6ff8b9f-hosts-file\") pod \"node-resolver-mvftd\" (UID: \"d4d297c3-08db-42da-8621-b9b3d6ff8b9f\") " pod="openshift-dns/node-resolver-mvftd" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703093 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/79c3cb22-0efb-42bc-8697-ad6431a2a874-kubelet-config\") pod \"global-pull-secret-syncer-x59k7\" (UID: \"79c3cb22-0efb-42bc-8697-ad6431a2a874\") " pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703060 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-sysctl-d\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703110 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpkg4\" (UniqueName: \"kubernetes.io/projected/35a33f42-b524-465b-bb4a-ab3526a6a6d9-kube-api-access-jpkg4\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.703388 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703132 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnqsd\" (UniqueName: \"kubernetes.io/projected/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-kube-api-access-wnqsd\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703143 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d4d297c3-08db-42da-8621-b9b3d6ff8b9f-hosts-file\") pod \"node-resolver-mvftd\" (UID: \"d4d297c3-08db-42da-8621-b9b3d6ff8b9f\") " pod="openshift-dns/node-resolver-mvftd" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703163 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-cni-netd\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703163 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93-serviceca\") pod \"node-ca-sgvv4\" (UID: \"51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93\") " pod="openshift-image-registry/node-ca-sgvv4" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703198 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-kubelet\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703205 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-cni-netd\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703265 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-kubelet\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703343 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/35a33f42-b524-465b-bb4a-ab3526a6a6d9-ovnkube-config\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703378 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703402 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-modprobe-d\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703422 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/79c3cb22-0efb-42bc-8697-ad6431a2a874-dbus\") pod \"global-pull-secret-syncer-x59k7\" (UID: \"79c3cb22-0efb-42bc-8697-ad6431a2a874\") " pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703444 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-etc-openvswitch\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703447 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703465 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-log-socket\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703499 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjsvx\" (UniqueName: \"kubernetes.io/projected/d4d297c3-08db-42da-8621-b9b3d6ff8b9f-kube-api-access-wjsvx\") pod \"node-resolver-mvftd\" (UID: \"d4d297c3-08db-42da-8621-b9b3d6ff8b9f\") " pod="openshift-dns/node-resolver-mvftd" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703522 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-var-lib-openvswitch\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703524 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-modprobe-d\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.704174 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703542 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/35a33f42-b524-465b-bb4a-ab3526a6a6d9-ovn-node-metrics-cert\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703500 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-etc-openvswitch\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703534 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5e788c00-eca3-4b87-98e5-89bd93f0abd0-konnectivity-ca\") pod \"konnectivity-agent-mgrlj\" (UID: \"5e788c00-eca3-4b87-98e5-89bd93f0abd0\") " pod="kube-system/konnectivity-agent-mgrlj" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703566 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/79c3cb22-0efb-42bc-8697-ad6431a2a874-dbus\") pod \"global-pull-secret-syncer-x59k7\" (UID: \"79c3cb22-0efb-42bc-8697-ad6431a2a874\") " pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703579 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-run-netns\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703597 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-var-lib-openvswitch\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703603 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-run-ovn\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703606 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-log-socket\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703641 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-slash\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703642 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-run-netns\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703664 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-cni-bin\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703675 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-run-ovn\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703712 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-slash\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703713 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-host-cni-bin\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703720 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-run\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703767 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-run-openvswitch\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703792 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/35a33f42-b524-465b-bb4a-ab3526a6a6d9-ovnkube-script-lib\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703811 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35a33f42-b524-465b-bb4a-ab3526a6a6d9-run-openvswitch\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703887 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-run\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703918 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/35a33f42-b524-465b-bb4a-ab3526a6a6d9-ovnkube-config\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703930 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-tuned\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703963 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbgnn\" (UniqueName: \"kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn\") pod \"network-check-target-vtb42\" (UID: \"f57dac14-ec36-409f-b8ef-e7a993c313ae\") " pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.703985 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93-host\") pod \"node-ca-sgvv4\" (UID: \"51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93\") " pod="openshift-image-registry/node-ca-sgvv4" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.704032 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93-host\") pod \"node-ca-sgvv4\" (UID: \"51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93\") " pod="openshift-image-registry/node-ca-sgvv4" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.704090 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/35a33f42-b524-465b-bb4a-ab3526a6a6d9-env-overrides\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.704164 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4d297c3-08db-42da-8621-b9b3d6ff8b9f-tmp-dir\") pod \"node-resolver-mvftd\" (UID: \"d4d297c3-08db-42da-8621-b9b3d6ff8b9f\") " pod="openshift-dns/node-resolver-mvftd" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.704188 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmj8v\" (UniqueName: \"kubernetes.io/projected/51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93-kube-api-access-kmj8v\") pod \"node-ca-sgvv4\" (UID: \"51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93\") " pod="openshift-image-registry/node-ca-sgvv4" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.704249 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-sysctl-conf\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.704283 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-lib-modules\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.704301 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-host\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.704320 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-tmp\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.704344 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/35a33f42-b524-465b-bb4a-ab3526a6a6d9-ovnkube-script-lib\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.704353 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret\") pod \"global-pull-secret-syncer-x59k7\" (UID: \"79c3cb22-0efb-42bc-8697-ad6431a2a874\") " pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.704491 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-sysctl-conf\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.704532 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/35a33f42-b524-465b-bb4a-ab3526a6a6d9-env-overrides\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.704563 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-lib-modules\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.705905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.704608 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-host\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.706483 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:25.704673 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 20:50:25.706483 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:25.704731 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret podName:79c3cb22-0efb-42bc-8697-ad6431a2a874 nodeName:}" failed. No retries permitted until 2026-09-09 20:50:26.204712074 +0000 UTC m=+3.043911183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret") pod "global-pull-secret-syncer-x59k7" (UID: "79c3cb22-0efb-42bc-8697-ad6431a2a874") : object "kube-system"/"original-pull-secret" not registered Sep 09 20:50:25.706483 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.704673 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4d297c3-08db-42da-8621-b9b3d6ff8b9f-tmp-dir\") pod \"node-resolver-mvftd\" (UID: \"d4d297c3-08db-42da-8621-b9b3d6ff8b9f\") " pod="openshift-dns/node-resolver-mvftd" Sep 09 20:50:25.706483 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.705738 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5e788c00-eca3-4b87-98e5-89bd93f0abd0-agent-certs\") pod \"konnectivity-agent-mgrlj\" (UID: \"5e788c00-eca3-4b87-98e5-89bd93f0abd0\") " pod="kube-system/konnectivity-agent-mgrlj" Sep 09 20:50:25.706483 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.706034 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/35a33f42-b524-465b-bb4a-ab3526a6a6d9-ovn-node-metrics-cert\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.706722 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.706693 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-etc-tuned\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.706778 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.706761 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-tmp\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.713046 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:25.712774 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 20:50:25.713046 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:25.712798 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 20:50:25.713046 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:25.712810 2562 projected.go:196] Error preparing data for projected volume kube-api-access-jbgnn for pod openshift-network-diagnostics/network-check-target-vtb42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:50:25.713046 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:25.712880 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn podName:f57dac14-ec36-409f-b8ef-e7a993c313ae nodeName:}" failed. No retries permitted until 2026-09-09 20:50:26.212861429 +0000 UTC m=+3.052060548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jbgnn" (UniqueName: "kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn") pod "network-check-target-vtb42" (UID: "f57dac14-ec36-409f-b8ef-e7a993c313ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:50:25.715342 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.715318 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjsvx\" (UniqueName: \"kubernetes.io/projected/d4d297c3-08db-42da-8621-b9b3d6ff8b9f-kube-api-access-wjsvx\") pod \"node-resolver-mvftd\" (UID: \"d4d297c3-08db-42da-8621-b9b3d6ff8b9f\") " pod="openshift-dns/node-resolver-mvftd" Sep 09 20:50:25.715559 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.715538 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmj8v\" (UniqueName: \"kubernetes.io/projected/51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93-kube-api-access-kmj8v\") pod \"node-ca-sgvv4\" (UID: \"51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93\") " pod="openshift-image-registry/node-ca-sgvv4" Sep 09 20:50:25.716056 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.716037 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnqsd\" (UniqueName: \"kubernetes.io/projected/aefdf0d5-977d-4de8-a2e3-3be5055e3f37-kube-api-access-wnqsd\") pod \"tuned-jkgsv\" (UID: \"aefdf0d5-977d-4de8-a2e3-3be5055e3f37\") " pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.716155 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.716117 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpkg4\" (UniqueName: \"kubernetes.io/projected/35a33f42-b524-465b-bb4a-ab3526a6a6d9-kube-api-access-jpkg4\") pod \"ovnkube-node-kjz44\" (UID: \"35a33f42-b524-465b-bb4a-ab3526a6a6d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:25.798574 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.798483 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4h5r5" Sep 09 20:50:25.806921 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.806900 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-prbnq" Sep 09 20:50:25.816337 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.816318 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" Sep 09 20:50:25.821774 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.821757 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-krlkq" Sep 09 20:50:25.831228 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.831206 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" Sep 09 20:50:25.838695 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.838675 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mgrlj" Sep 09 20:50:25.846110 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.846088 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mvftd" Sep 09 20:50:25.852514 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.852495 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sgvv4" Sep 09 20:50:25.858853 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:25.858836 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:26.011686 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:26.011662 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 20:50:26.106332 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:26.106261 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs\") pod \"network-metrics-daemon-lqd29\" (UID: \"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5\") " pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:26.106478 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:26.106398 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:50:26.106478 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:26.106460 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs podName:ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5 nodeName:}" failed. No retries permitted until 2026-09-09 20:50:27.106443902 +0000 UTC m=+3.945643022 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs") pod "network-metrics-daemon-lqd29" (UID: "ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:50:26.207537 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:26.207496 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret\") pod \"global-pull-secret-syncer-x59k7\" (UID: \"79c3cb22-0efb-42bc-8697-ad6431a2a874\") " pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:26.207704 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:26.207647 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 20:50:26.207763 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:26.207713 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret podName:79c3cb22-0efb-42bc-8697-ad6431a2a874 nodeName:}" failed. No retries permitted until 2026-09-09 20:50:27.207696316 +0000 UTC m=+4.046895435 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret") pod "global-pull-secret-syncer-x59k7" (UID: "79c3cb22-0efb-42bc-8697-ad6431a2a874") : object "kube-system"/"original-pull-secret" not registered Sep 09 20:50:26.307959 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:26.307929 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbgnn\" (UniqueName: \"kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn\") pod \"network-check-target-vtb42\" (UID: \"f57dac14-ec36-409f-b8ef-e7a993c313ae\") " pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:26.308088 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:26.308036 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 20:50:26.308088 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:26.308051 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 20:50:26.308088 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:26.308059 2562 projected.go:196] Error preparing data for projected volume kube-api-access-jbgnn for pod openshift-network-diagnostics/network-check-target-vtb42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:50:26.308185 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:26.308104 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn podName:f57dac14-ec36-409f-b8ef-e7a993c313ae nodeName:}" failed. No retries permitted until 2026-09-09 20:50:27.3080929 +0000 UTC m=+4.147292018 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbgnn" (UniqueName: "kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn") pod "network-check-target-vtb42" (UID: "f57dac14-ec36-409f-b8ef-e7a993c313ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:50:26.529909 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:26.529838 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:45:25 +0000 UTC" deadline="2028-02-23 08:23:29.196386481 +0000 UTC" Sep 09 20:50:26.529909 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:26.529871 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12755h33m2.666519966s" Sep 09 20:50:26.626068 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:26.625974 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a6278fc_dcb1_4e6f_9a0f_7805e09ccde7.slice/crio-0d6f11731d77c42082b014a04ad1003b3715adeabbb80e288e0f4a607dfabc56 WatchSource:0}: Error finding container 0d6f11731d77c42082b014a04ad1003b3715adeabbb80e288e0f4a607dfabc56: Status 404 returned error can't find the container with id 0d6f11731d77c42082b014a04ad1003b3715adeabbb80e288e0f4a607dfabc56 Sep 09 20:50:26.628964 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:26.628944 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e788c00_eca3_4b87_98e5_89bd93f0abd0.slice/crio-59365a4e02f908bc3623943c65cfc9c79146a554d882ce4af9e59a05174d8d35 WatchSource:0}: Error finding container 59365a4e02f908bc3623943c65cfc9c79146a554d882ce4af9e59a05174d8d35: Status 404 returned error can't find the container with id 59365a4e02f908bc3623943c65cfc9c79146a554d882ce4af9e59a05174d8d35 Sep 09 20:50:26.630928 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:26.630883 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f7991ef_372f_4d93_b7b8_25c7bad0528d.slice/crio-81e664d928e743effcaf331a9fa658ee152cd6936cca399d36a6b735349743c3 WatchSource:0}: Error finding container 81e664d928e743effcaf331a9fa658ee152cd6936cca399d36a6b735349743c3: Status 404 returned error can't find the container with id 81e664d928e743effcaf331a9fa658ee152cd6936cca399d36a6b735349743c3 Sep 09 20:50:26.632371 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:26.632348 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4d297c3_08db_42da_8621_b9b3d6ff8b9f.slice/crio-cf1fe83ccf3e62e3a5f091f3123d98a2cca7f5c1cd4207174aabdc81fe9b119e WatchSource:0}: Error finding container cf1fe83ccf3e62e3a5f091f3123d98a2cca7f5c1cd4207174aabdc81fe9b119e: Status 404 returned error can't find the container with id cf1fe83ccf3e62e3a5f091f3123d98a2cca7f5c1cd4207174aabdc81fe9b119e Sep 09 20:50:26.641900 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:26.641876 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mgrlj" event={"ID":"5e788c00-eca3-4b87-98e5-89bd93f0abd0","Type":"ContainerStarted","Data":"59365a4e02f908bc3623943c65cfc9c79146a554d882ce4af9e59a05174d8d35"} Sep 09 20:50:26.642917 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:26.642896 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4h5r5" event={"ID":"9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7","Type":"ContainerStarted","Data":"0d6f11731d77c42082b014a04ad1003b3715adeabbb80e288e0f4a607dfabc56"} Sep 09 20:50:26.643840 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:26.643819 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" event={"ID":"5f7991ef-372f-4d93-b7b8-25c7bad0528d","Type":"ContainerStarted","Data":"81e664d928e743effcaf331a9fa658ee152cd6936cca399d36a6b735349743c3"} Sep 09 20:50:26.644849 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:26.644832 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mvftd" event={"ID":"d4d297c3-08db-42da-8621-b9b3d6ff8b9f","Type":"ContainerStarted","Data":"cf1fe83ccf3e62e3a5f091f3123d98a2cca7f5c1cd4207174aabdc81fe9b119e"} Sep 09 20:50:26.655411 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:26.655387 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode26cba9c_5ba0_44c8_9d41_c63dad722bbe.slice/crio-7a921f0f9653ef8691567c80bc0259bbe7cb1c9d02e7010c2fe92a8273c9e735 WatchSource:0}: Error finding container 7a921f0f9653ef8691567c80bc0259bbe7cb1c9d02e7010c2fe92a8273c9e735: Status 404 returned error can't find the container with id 7a921f0f9653ef8691567c80bc0259bbe7cb1c9d02e7010c2fe92a8273c9e735 Sep 09 20:50:26.655944 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:26.655921 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51ddcfe5_9a4d_4fe5_96a8_ea3ffbf63f93.slice/crio-7a0848ea3678ee441a618c6c101e1164a8bfb41d65cf081cb1f0cb60b04f6492 WatchSource:0}: Error finding container 7a0848ea3678ee441a618c6c101e1164a8bfb41d65cf081cb1f0cb60b04f6492: Status 404 returned error can't find the container with id 7a0848ea3678ee441a618c6c101e1164a8bfb41d65cf081cb1f0cb60b04f6492 Sep 09 20:50:26.657118 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:26.656962 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46ffa1a9_4a22_4274_b366_30bb80a2fa85.slice/crio-1247dff990dfbbc9be0c951bba6139b5a94b504c7fe6fafab92f4e39aa50bb7e WatchSource:0}: Error finding container 1247dff990dfbbc9be0c951bba6139b5a94b504c7fe6fafab92f4e39aa50bb7e: Status 404 returned error can't find the container with id 1247dff990dfbbc9be0c951bba6139b5a94b504c7fe6fafab92f4e39aa50bb7e Sep 09 20:50:26.657737 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:26.657673 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaefdf0d5_977d_4de8_a2e3_3be5055e3f37.slice/crio-254a5a06a2a3bd480321223702935017762051af0a0a2644718a250fdec0f1e7 WatchSource:0}: Error finding container 254a5a06a2a3bd480321223702935017762051af0a0a2644718a250fdec0f1e7: Status 404 returned error can't find the container with id 254a5a06a2a3bd480321223702935017762051af0a0a2644718a250fdec0f1e7 Sep 09 20:50:26.658838 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:26.658820 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35a33f42_b524_465b_bb4a_ab3526a6a6d9.slice/crio-c66a2df30d00f26394000d6fc08a448e3ba35512ce50cf4de19d63704a9b0244 WatchSource:0}: Error finding container c66a2df30d00f26394000d6fc08a448e3ba35512ce50cf4de19d63704a9b0244: Status 404 returned error can't find the container with id c66a2df30d00f26394000d6fc08a448e3ba35512ce50cf4de19d63704a9b0244 Sep 09 20:50:27.112864 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:27.112785 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs\") pod \"network-metrics-daemon-lqd29\" (UID: \"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5\") " pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:27.113010 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:27.112892 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:50:27.113010 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:27.112938 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs podName:ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5 nodeName:}" failed. No retries permitted until 2026-09-09 20:50:29.112926633 +0000 UTC m=+5.952125739 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs") pod "network-metrics-daemon-lqd29" (UID: "ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:50:27.214135 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:27.214096 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret\") pod \"global-pull-secret-syncer-x59k7\" (UID: \"79c3cb22-0efb-42bc-8697-ad6431a2a874\") " pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:27.214312 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:27.214261 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 20:50:27.214385 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:27.214326 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret podName:79c3cb22-0efb-42bc-8697-ad6431a2a874 nodeName:}" failed. No retries permitted until 2026-09-09 20:50:29.214308808 +0000 UTC m=+6.053507941 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret") pod "global-pull-secret-syncer-x59k7" (UID: "79c3cb22-0efb-42bc-8697-ad6431a2a874") : object "kube-system"/"original-pull-secret" not registered Sep 09 20:50:27.315888 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:27.315193 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbgnn\" (UniqueName: \"kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn\") pod \"network-check-target-vtb42\" (UID: \"f57dac14-ec36-409f-b8ef-e7a993c313ae\") " pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:27.315888 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:27.315437 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 20:50:27.315888 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:27.315456 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 20:50:27.315888 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:27.315469 2562 projected.go:196] Error preparing data for projected volume kube-api-access-jbgnn for pod openshift-network-diagnostics/network-check-target-vtb42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:50:27.315888 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:27.315548 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn podName:f57dac14-ec36-409f-b8ef-e7a993c313ae nodeName:}" failed. No retries permitted until 2026-09-09 20:50:29.315522761 +0000 UTC m=+6.154721893 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbgnn" (UniqueName: "kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn") pod "network-check-target-vtb42" (UID: "f57dac14-ec36-409f-b8ef-e7a993c313ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:50:27.531141 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:27.531019 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:45:25 +0000 UTC" deadline="2028-04-07 01:17:27.290130916 +0000 UTC" Sep 09 20:50:27.531141 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:27.531080 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13804h26m59.759054712s" Sep 09 20:50:27.636075 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:27.635270 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:27.636075 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:27.635410 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:27.636075 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:27.635785 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:27.636075 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:27.635875 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:27.636075 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:27.635930 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:27.636075 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:27.635998 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:27.656659 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:27.656597 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-krlkq" event={"ID":"46ffa1a9-4a22-4274-b366-30bb80a2fa85","Type":"ContainerStarted","Data":"1247dff990dfbbc9be0c951bba6139b5a94b504c7fe6fafab92f4e39aa50bb7e"} Sep 09 20:50:27.667128 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:27.667033 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sgvv4" event={"ID":"51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93","Type":"ContainerStarted","Data":"7a0848ea3678ee441a618c6c101e1164a8bfb41d65cf081cb1f0cb60b04f6492"} Sep 09 20:50:27.672176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:27.672082 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prbnq" event={"ID":"e26cba9c-5ba0-44c8-9d41-c63dad722bbe","Type":"ContainerStarted","Data":"7a921f0f9653ef8691567c80bc0259bbe7cb1c9d02e7010c2fe92a8273c9e735"} Sep 09 20:50:27.676828 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:27.676762 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" event={"ID":"aefdf0d5-977d-4de8-a2e3-3be5055e3f37","Type":"ContainerStarted","Data":"254a5a06a2a3bd480321223702935017762051af0a0a2644718a250fdec0f1e7"} Sep 09 20:50:27.680893 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:27.680334 2562 generic.go:356] "Generic (PLEG): container finished" podID="c92fd857038978a287ea82826bec9300" containerID="f0dea94c347a5d7793c082520d0521ffeb563c3e51542d07e1b9331fcee37894" exitCode=0 Sep 09 20:50:27.680893 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:27.680422 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal" event={"ID":"c92fd857038978a287ea82826bec9300","Type":"ContainerDied","Data":"f0dea94c347a5d7793c082520d0521ffeb563c3e51542d07e1b9331fcee37894"} Sep 09 20:50:27.689462 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:27.689103 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-83.ec2.internal" event={"ID":"2df482739528dd8ed1304969d8f65a5c","Type":"ContainerStarted","Data":"cf247d98924f9761fc90178563ffd1fa26277a2cbbf7148d3846ad9a72c2172b"} Sep 09 20:50:27.695711 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:27.695613 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" event={"ID":"35a33f42-b524-465b-bb4a-ab3526a6a6d9","Type":"ContainerStarted","Data":"c66a2df30d00f26394000d6fc08a448e3ba35512ce50cf4de19d63704a9b0244"} Sep 09 20:50:28.719747 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:28.719647 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal" event={"ID":"c92fd857038978a287ea82826bec9300","Type":"ContainerStarted","Data":"4d3520c0f52f3b60071724807235786221b1b1e7e0eb753571b99556a9d87e98"} Sep 09 20:50:28.737673 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:28.737602 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-83.ec2.internal" podStartSLOduration=4.736868666 podStartE2EDuration="4.736868666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:50:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:50:27.714062992 +0000 UTC m=+4.553262125" watchObservedRunningTime="2026-09-09 20:50:28.736868666 +0000 UTC m=+5.576067797" Sep 09 20:50:28.737974 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:28.737919 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-83.ec2.internal" podStartSLOduration=4.737905348 podStartE2EDuration="4.737905348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:50:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:50:28.737752419 +0000 UTC m=+5.576951539" watchObservedRunningTime="2026-09-09 20:50:28.737905348 +0000 UTC m=+5.577104479" Sep 09 20:50:29.132199 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:29.131609 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs\") pod \"network-metrics-daemon-lqd29\" (UID: \"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5\") " pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:29.132199 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:29.131748 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:50:29.132199 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:29.131814 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs podName:ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5 nodeName:}" failed. No retries permitted until 2026-09-09 20:50:33.131796665 +0000 UTC m=+9.970995778 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs") pod "network-metrics-daemon-lqd29" (UID: "ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:50:29.233576 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:29.232845 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret\") pod \"global-pull-secret-syncer-x59k7\" (UID: \"79c3cb22-0efb-42bc-8697-ad6431a2a874\") " pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:29.233576 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:29.233019 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 20:50:29.233576 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:29.233082 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret podName:79c3cb22-0efb-42bc-8697-ad6431a2a874 nodeName:}" failed. No retries permitted until 2026-09-09 20:50:33.233065277 +0000 UTC m=+10.072264399 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret") pod "global-pull-secret-syncer-x59k7" (UID: "79c3cb22-0efb-42bc-8697-ad6431a2a874") : object "kube-system"/"original-pull-secret" not registered Sep 09 20:50:29.334650 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:29.334067 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbgnn\" (UniqueName: \"kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn\") pod \"network-check-target-vtb42\" (UID: \"f57dac14-ec36-409f-b8ef-e7a993c313ae\") " pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:29.334650 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:29.334227 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 20:50:29.334650 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:29.334266 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 20:50:29.334650 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:29.334281 2562 projected.go:196] Error preparing data for projected volume kube-api-access-jbgnn for pod openshift-network-diagnostics/network-check-target-vtb42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:50:29.334650 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:29.334340 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn podName:f57dac14-ec36-409f-b8ef-e7a993c313ae nodeName:}" failed. No retries permitted until 2026-09-09 20:50:33.334322202 +0000 UTC m=+10.173521327 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbgnn" (UniqueName: "kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn") pod "network-check-target-vtb42" (UID: "f57dac14-ec36-409f-b8ef-e7a993c313ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:50:29.634370 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:29.634333 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:29.634550 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:29.634477 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:29.634915 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:29.634894 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:29.635037 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:29.635016 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:29.635176 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:29.635161 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:29.635287 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:29.635263 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:31.635993 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:31.635172 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:31.635993 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:31.635318 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:31.635993 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:31.635723 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:31.635993 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:31.635824 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:31.635993 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:31.635878 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:31.635993 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:31.635945 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:33.166445 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:33.166189 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs\") pod \"network-metrics-daemon-lqd29\" (UID: \"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5\") " pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:33.166897 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:33.166626 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:50:33.166897 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:33.166683 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs podName:ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5 nodeName:}" failed. No retries permitted until 2026-09-09 20:50:41.166669564 +0000 UTC m=+18.005868671 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs") pod "network-metrics-daemon-lqd29" (UID: "ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:50:33.267509 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:33.267460 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret\") pod \"global-pull-secret-syncer-x59k7\" (UID: \"79c3cb22-0efb-42bc-8697-ad6431a2a874\") " pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:33.267688 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:33.267604 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 20:50:33.267688 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:33.267668 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret podName:79c3cb22-0efb-42bc-8697-ad6431a2a874 nodeName:}" failed. No retries permitted until 2026-09-09 20:50:41.267649849 +0000 UTC m=+18.106848971 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret") pod "global-pull-secret-syncer-x59k7" (UID: "79c3cb22-0efb-42bc-8697-ad6431a2a874") : object "kube-system"/"original-pull-secret" not registered Sep 09 20:50:33.367864 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:33.367823 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbgnn\" (UniqueName: \"kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn\") pod \"network-check-target-vtb42\" (UID: \"f57dac14-ec36-409f-b8ef-e7a993c313ae\") " pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:33.368033 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:33.368006 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 20:50:33.368543 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:33.368029 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 20:50:33.368543 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:33.368107 2562 projected.go:196] Error preparing data for projected volume kube-api-access-jbgnn for pod openshift-network-diagnostics/network-check-target-vtb42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:50:33.368543 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:33.368167 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn podName:f57dac14-ec36-409f-b8ef-e7a993c313ae nodeName:}" failed. No retries permitted until 2026-09-09 20:50:41.368149637 +0000 UTC m=+18.207348750 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbgnn" (UniqueName: "kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn") pod "network-check-target-vtb42" (UID: "f57dac14-ec36-409f-b8ef-e7a993c313ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:50:33.634443 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:33.634330 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:33.634611 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:33.634442 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:33.634814 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:33.634796 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:33.634896 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:33.634880 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:33.634953 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:33.634929 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:33.635007 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:33.634980 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:35.635767 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:35.634979 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:35.635767 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:35.635097 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:35.635767 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:35.635516 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:35.635767 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:35.635612 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:35.635767 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:35.635668 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:35.635767 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:35.635720 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:37.633031 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:37.632939 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:37.633502 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:37.633082 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:37.633502 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:37.633099 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:37.633502 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:37.633096 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:37.633502 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:37.633171 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:37.633502 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:37.633228 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:39.632586 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:39.632548 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:39.632586 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:39.632581 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:39.633055 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:39.632548 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:39.633055 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:39.632697 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:39.635948 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:39.633192 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:39.635948 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:39.633209 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:41.227280 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:41.227225 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs\") pod \"network-metrics-daemon-lqd29\" (UID: \"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5\") " pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:41.227677 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:41.227349 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:50:41.227677 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:41.227415 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs podName:ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5 nodeName:}" failed. No retries permitted until 2026-09-09 20:50:57.227398189 +0000 UTC m=+34.066597297 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs") pod "network-metrics-daemon-lqd29" (UID: "ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:50:41.328542 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:41.328453 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret\") pod \"global-pull-secret-syncer-x59k7\" (UID: \"79c3cb22-0efb-42bc-8697-ad6431a2a874\") " pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:41.328693 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:41.328646 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 20:50:41.328743 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:41.328728 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret podName:79c3cb22-0efb-42bc-8697-ad6431a2a874 nodeName:}" failed. No retries permitted until 2026-09-09 20:50:57.328709075 +0000 UTC m=+34.167908187 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret") pod "global-pull-secret-syncer-x59k7" (UID: "79c3cb22-0efb-42bc-8697-ad6431a2a874") : object "kube-system"/"original-pull-secret" not registered Sep 09 20:50:41.429270 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:41.429221 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbgnn\" (UniqueName: \"kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn\") pod \"network-check-target-vtb42\" (UID: \"f57dac14-ec36-409f-b8ef-e7a993c313ae\") " pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:41.429439 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:41.429388 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 20:50:41.429439 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:41.429418 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 20:50:41.429439 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:41.429432 2562 projected.go:196] Error preparing data for projected volume kube-api-access-jbgnn for pod openshift-network-diagnostics/network-check-target-vtb42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:50:41.429576 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:41.429500 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn podName:f57dac14-ec36-409f-b8ef-e7a993c313ae nodeName:}" failed. No retries permitted until 2026-09-09 20:50:57.429480353 +0000 UTC m=+34.268679471 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbgnn" (UniqueName: "kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn") pod "network-check-target-vtb42" (UID: "f57dac14-ec36-409f-b8ef-e7a993c313ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:50:41.632790 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:41.632707 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:41.632955 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:41.632710 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:41.632955 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:41.632838 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:41.632955 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:41.632710 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:41.632955 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:41.632943 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:41.633169 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:41.632989 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:43.633447 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:43.633416 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:43.633961 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:43.633523 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:43.633961 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:43.633599 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:43.633961 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:43.633700 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:43.633961 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:43.633742 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:43.633961 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:43.633808 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:44.757058 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:44.756846 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" event={"ID":"aefdf0d5-977d-4de8-a2e3-3be5055e3f37","Type":"ContainerStarted","Data":"bae384bf8ed9d9f63eb80f1e74b3511c78594bd0b97e16152ebf0a2517ebae2f"} Sep 09 20:50:44.765305 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:44.765275 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" event={"ID":"35a33f42-b524-465b-bb4a-ab3526a6a6d9","Type":"ContainerStarted","Data":"9e70244da7c7af667d5ec2bfa171dbe561825b8ff7d05c2e7e03575fe6c34e5e"} Sep 09 20:50:44.783166 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:44.782793 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jkgsv" podStartSLOduration=3.870505417 podStartE2EDuration="21.782770363s" totalImagesPullingTime="17.912264946s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:50:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:50:44.78161458 +0000 UTC m=+21.620813712" watchObservedRunningTime="2026-09-09 20:50:44.782770363 +0000 UTC m=+21.621969496" Sep 09 20:50:44.804034 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:44.801809 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-krlkq" podStartSLOduration=3.835956811 podStartE2EDuration="21.801785141s" totalImagesPullingTime="17.96582833s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:50:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:50:44.800667355 +0000 UTC m=+21.639866487" watchObservedRunningTime="2026-09-09 20:50:44.801785141 +0000 UTC m=+21.640984272" Sep 09 20:50:45.633794 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.633562 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:45.633972 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.633562 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:45.633972 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:45.633912 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:45.634071 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.633590 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:45.634071 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:45.634015 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:45.634163 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:45.634088 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:45.772448 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.772409 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sgvv4" event={"ID":"51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93","Type":"ContainerStarted","Data":"ad5f199c8b681acaf6855a85b49d23d2970aaa1dfbb39d81603d7bde9ea1bc38"} Sep 09 20:50:45.773978 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.773953 2562 generic.go:356] "Generic (PLEG): container finished" podID="e26cba9c-5ba0-44c8-9d41-c63dad722bbe" containerID="e7ffd7ce4de5b9e514d2a692f62ee50d8fa9f12e56de994190e0c15cd0aaf6de" exitCode=0 Sep 09 20:50:45.774079 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.774018 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prbnq" event={"ID":"e26cba9c-5ba0-44c8-9d41-c63dad722bbe","Type":"ContainerDied","Data":"e7ffd7ce4de5b9e514d2a692f62ee50d8fa9f12e56de994190e0c15cd0aaf6de"} Sep 09 20:50:45.775628 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.775599 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mvftd" event={"ID":"d4d297c3-08db-42da-8621-b9b3d6ff8b9f","Type":"ContainerStarted","Data":"5e6727dea60c1861078c973a2cc861d32afacecbf166ea82fdd0ed6b417d99b0"} Sep 09 20:50:45.776957 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.776940 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" event={"ID":"5f7991ef-372f-4d93-b7b8-25c7bad0528d","Type":"ContainerStarted","Data":"cb8bdd08c7319de871c381633ba50a3306c07c61fcd42f0ddc2fb0d540cc4671"} Sep 09 20:50:45.778342 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.778322 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mgrlj" event={"ID":"5e788c00-eca3-4b87-98e5-89bd93f0abd0","Type":"ContainerStarted","Data":"44b4bf61738447396bdf6d30ee678d627d0b26c6ff2e3f03c4a40f399564d0bb"} Sep 09 20:50:45.781324 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.781305 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" event={"ID":"35a33f42-b524-465b-bb4a-ab3526a6a6d9","Type":"ContainerStarted","Data":"f19fc4f82277818a1d9d3c0273269ec833f3125ba976386f0eef35e81744d2cf"} Sep 09 20:50:45.781406 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.781367 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" event={"ID":"35a33f42-b524-465b-bb4a-ab3526a6a6d9","Type":"ContainerStarted","Data":"bf0bc1f32be22562a8792c928d9901739666bdc69782ae1edcfcee318c5e6200"} Sep 09 20:50:45.781455 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.781423 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" event={"ID":"35a33f42-b524-465b-bb4a-ab3526a6a6d9","Type":"ContainerStarted","Data":"1d9b778c158d66426ec16e8e3a3a9dd5194b2b343d86b3cbd577d52c8f39ac91"} Sep 09 20:50:45.781489 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.781473 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" event={"ID":"35a33f42-b524-465b-bb4a-ab3526a6a6d9","Type":"ContainerStarted","Data":"859fc5272560fdff235b8508322ea0f64aeaf15244be934a8a966aaa2ff50579"} Sep 09 20:50:45.781576 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.781521 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" event={"ID":"35a33f42-b524-465b-bb4a-ab3526a6a6d9","Type":"ContainerStarted","Data":"9626e93f28dfc8423988988712b60aacbf04676a14ad3f08bf9220341489cd34"} Sep 09 20:50:45.782658 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.782640 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-krlkq" event={"ID":"46ffa1a9-4a22-4274-b366-30bb80a2fa85","Type":"ContainerStarted","Data":"4bdb0e1e66420eef4bfcd5e0261f94419d5c8126f46232e94625974d7a53bcba"} Sep 09 20:50:45.787063 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.787023 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sgvv4" podStartSLOduration=4.871315931 podStartE2EDuration="22.787010168s" totalImagesPullingTime="17.915694237s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:50:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:50:45.786591697 +0000 UTC m=+22.625790827" watchObservedRunningTime="2026-09-09 20:50:45.787010168 +0000 UTC m=+22.626209298" Sep 09 20:50:45.801289 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.801228 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mgrlj" podStartSLOduration=8.414897989 podStartE2EDuration="22.801212703s" totalImagesPullingTime="14.386314714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:50:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:50:45.801009295 +0000 UTC m=+22.640208437" watchObservedRunningTime="2026-09-09 20:50:45.801212703 +0000 UTC m=+22.640411833" Sep 09 20:50:45.815511 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:45.815475 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mvftd" podStartSLOduration=4.896001895 podStartE2EDuration="22.815461405s" totalImagesPullingTime="17.91945951s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:50:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:50:45.815358496 +0000 UTC m=+22.654557626" watchObservedRunningTime="2026-09-09 20:50:45.815461405 +0000 UTC m=+22.654660529" Sep 09 20:50:46.116016 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:46.115867 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 09 20:50:46.562425 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:46.562313 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T20:50:46.116010094Z","UUID":"801c8286-eb75-4025-b809-ca9f5386e9d8","Handler":null,"Name":"","Endpoint":""} Sep 09 20:50:46.564175 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:46.564151 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 09 20:50:46.564318 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:46.564189 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 09 20:50:46.787099 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:46.787059 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4h5r5" event={"ID":"9a6278fc-dcb1-4e6f-9a0f-7805e09ccde7","Type":"ContainerStarted","Data":"86c220d82334856295360cc2dd96684b442960425cbdce87ec2f7a105871bf40"} Sep 09 20:50:46.789890 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:46.789818 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" event={"ID":"5f7991ef-372f-4d93-b7b8-25c7bad0528d","Type":"ContainerStarted","Data":"b966e2e936db442c2b9cc78a39c78a125395673c27703e88ef78d439ba04f43f"} Sep 09 20:50:46.802180 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:46.802129 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4h5r5" podStartSLOduration=5.857530587 podStartE2EDuration="23.802110819s" totalImagesPullingTime="17.944580232s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:50:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:50:46.801860747 +0000 UTC m=+23.641059877" watchObservedRunningTime="2026-09-09 20:50:46.802110819 +0000 UTC m=+23.641309951" Sep 09 20:50:47.636784 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:47.636760 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:47.636902 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:47.636871 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:47.636902 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:47.636879 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:47.636902 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:47.636901 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:47.637072 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:47.636976 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:47.637124 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:47.637068 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:47.793374 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:47.793339 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" event={"ID":"5f7991ef-372f-4d93-b7b8-25c7bad0528d","Type":"ContainerStarted","Data":"42f38a9cf1353058a13a793b19deb15af1dd5fee78998e453e5b8dbb7cbc2927"} Sep 09 20:50:47.796954 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:47.796926 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" event={"ID":"35a33f42-b524-465b-bb4a-ab3526a6a6d9","Type":"ContainerStarted","Data":"cf98ffde019e88821e362c5cdf440ea89d9a41a997483ea4cde858e7146c84d9"} Sep 09 20:50:47.811733 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:47.811691 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rss78" podStartSLOduration=4.2234926250000004 podStartE2EDuration="24.811677428s" totalImagesPullingTime="20.588184803s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:50:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:50:47.811585874 +0000 UTC m=+24.650785004" watchObservedRunningTime="2026-09-09 20:50:47.811677428 +0000 UTC m=+24.650876556" Sep 09 20:50:49.151917 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:49.151874 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mgrlj" Sep 09 20:50:49.152547 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:49.152527 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mgrlj" Sep 09 20:50:49.633329 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:49.633291 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:49.633531 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:49.633295 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:49.633531 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:49.633448 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:49.633531 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:49.633291 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:49.633531 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:49.633501 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:49.633711 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:49.633596 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:49.801852 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:49.801823 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mgrlj" Sep 09 20:50:49.802383 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:49.802357 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mgrlj" Sep 09 20:50:50.808629 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:50.808413 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" event={"ID":"35a33f42-b524-465b-bb4a-ab3526a6a6d9","Type":"ContainerStarted","Data":"f8c8c24715bf08618e0485bda02a747f83dd6f2ef22249a5a2d2158fec4f7d6a"} Sep 09 20:50:50.809065 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:50.808691 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:50.810607 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:50.810580 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prbnq" event={"ID":"e26cba9c-5ba0-44c8-9d41-c63dad722bbe","Type":"ContainerStarted","Data":"31a0468206339dc19c7b6255b239c1fce41e1edef15419bda7677615874f70e9"} Sep 09 20:50:50.824887 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:50.824863 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:50.852283 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:50.851944 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" podStartSLOduration=9.886843758 podStartE2EDuration="27.851924607s" totalImagesPullingTime="17.965080849s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:50:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:50:50.848221479 +0000 UTC m=+27.687420601" watchObservedRunningTime="2026-09-09 20:50:50.851924607 +0000 UTC m=+27.691123717" Sep 09 20:50:51.632647 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:51.632599 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:51.632647 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:51.632641 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:51.632967 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:51.632739 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:51.632967 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:51.632608 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:51.632967 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:51.632879 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:51.633128 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:51.632978 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:51.815129 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:51.815090 2562 generic.go:356] "Generic (PLEG): container finished" podID="e26cba9c-5ba0-44c8-9d41-c63dad722bbe" containerID="31a0468206339dc19c7b6255b239c1fce41e1edef15419bda7677615874f70e9" exitCode=0 Sep 09 20:50:51.815617 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:51.815138 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prbnq" event={"ID":"e26cba9c-5ba0-44c8-9d41-c63dad722bbe","Type":"ContainerDied","Data":"31a0468206339dc19c7b6255b239c1fce41e1edef15419bda7677615874f70e9"} Sep 09 20:50:51.816305 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:51.815785 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:51.816305 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:51.815856 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:51.831796 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:51.831774 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:50:52.232624 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:52.232589 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x59k7"] Sep 09 20:50:52.232830 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:52.232721 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:52.232880 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:52.232829 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:52.235416 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:52.235390 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lqd29"] Sep 09 20:50:52.235568 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:52.235483 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:52.235568 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:52.235558 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:52.245030 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:52.244975 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vtb42"] Sep 09 20:50:52.245132 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:52.245050 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:52.245132 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:52.245116 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:53.634553 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:53.634323 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:53.635045 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:53.634377 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:53.635045 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:53.634660 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:53.635045 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:53.634692 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:53.822099 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:53.821992 2562 generic.go:356] "Generic (PLEG): container finished" podID="e26cba9c-5ba0-44c8-9d41-c63dad722bbe" containerID="d450153701c4712cea64f2f51937e04e3f23e97eb17785501178426dc218816e" exitCode=0 Sep 09 20:50:53.822099 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:53.822048 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prbnq" event={"ID":"e26cba9c-5ba0-44c8-9d41-c63dad722bbe","Type":"ContainerDied","Data":"d450153701c4712cea64f2f51937e04e3f23e97eb17785501178426dc218816e"} Sep 09 20:50:54.632634 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:54.632600 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:54.632792 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:54.632710 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:54.829254 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:54.829209 2562 generic.go:356] "Generic (PLEG): container finished" podID="e26cba9c-5ba0-44c8-9d41-c63dad722bbe" containerID="e7cc0e9f82eca0b5f55aadd0cabbc5caa6aebda9c97464a2173fec19dccf710b" exitCode=0 Sep 09 20:50:54.829609 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:54.829270 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prbnq" event={"ID":"e26cba9c-5ba0-44c8-9d41-c63dad722bbe","Type":"ContainerDied","Data":"e7cc0e9f82eca0b5f55aadd0cabbc5caa6aebda9c97464a2173fec19dccf710b"} Sep 09 20:50:55.633387 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:55.633350 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:55.633599 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:55.633470 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vtb42" podUID="f57dac14-ec36-409f-b8ef-e7a993c313ae" Sep 09 20:50:55.633599 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:55.633536 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:55.633714 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:55.633643 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:50:56.633342 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:56.633315 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:56.633797 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:56.633433 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x59k7" podUID="79c3cb22-0efb-42bc-8697-ad6431a2a874" Sep 09 20:50:57.245661 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.245623 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs\") pod \"network-metrics-daemon-lqd29\" (UID: \"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5\") " pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:57.245847 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:57.245794 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:50:57.245896 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:57.245868 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs podName:ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5 nodeName:}" failed. No retries permitted until 2026-09-09 20:51:29.245850672 +0000 UTC m=+66.085049784 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs") pod "network-metrics-daemon-lqd29" (UID: "ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 20:50:57.340039 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.340010 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-83.ec2.internal" event="NodeReady" Sep 09 20:50:57.340210 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.340177 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 09 20:50:57.346368 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.346343 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret\") pod \"global-pull-secret-syncer-x59k7\" (UID: \"79c3cb22-0efb-42bc-8697-ad6431a2a874\") " pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:57.346519 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:57.346457 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 20:50:57.346564 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:57.346519 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret podName:79c3cb22-0efb-42bc-8697-ad6431a2a874 nodeName:}" failed. No retries permitted until 2026-09-09 20:51:29.346499827 +0000 UTC m=+66.185698956 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret") pod "global-pull-secret-syncer-x59k7" (UID: "79c3cb22-0efb-42bc-8697-ad6431a2a874") : object "kube-system"/"original-pull-secret" not registered Sep 09 20:50:57.424869 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.424783 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8wlxq"] Sep 09 20:50:57.429960 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.429937 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8wlxq" Sep 09 20:50:57.431638 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.431612 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-crv44"] Sep 09 20:50:57.432943 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.432918 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 09 20:50:57.433173 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.433159 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 09 20:50:57.433991 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.433972 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wtvxz\"" Sep 09 20:50:57.434837 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.434811 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-crv44" Sep 09 20:50:57.439862 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.439837 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 09 20:50:57.439983 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.439874 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 09 20:50:57.439983 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.439911 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-z287q\"" Sep 09 20:50:57.440096 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.440028 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 09 20:50:57.442941 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.442919 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8wlxq"] Sep 09 20:50:57.447010 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.446982 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbgnn\" (UniqueName: \"kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn\") pod \"network-check-target-vtb42\" (UID: \"f57dac14-ec36-409f-b8ef-e7a993c313ae\") " pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:57.447138 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:57.447123 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 20:50:57.447204 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:57.447143 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 20:50:57.447204 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:57.447152 2562 projected.go:196] Error preparing data for projected volume kube-api-access-jbgnn for pod openshift-network-diagnostics/network-check-target-vtb42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:50:57.447204 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:57.447192 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn podName:f57dac14-ec36-409f-b8ef-e7a993c313ae nodeName:}" failed. No retries permitted until 2026-09-09 20:51:29.447180743 +0000 UTC m=+66.286379851 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbgnn" (UniqueName: "kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn") pod "network-check-target-vtb42" (UID: "f57dac14-ec36-409f-b8ef-e7a993c313ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 20:50:57.453774 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.453754 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-crv44"] Sep 09 20:50:57.547464 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.547423 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/df144005-7adc-4de3-8c5b-37584f6ba534-config-volume\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:50:57.547629 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.547488 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df144005-7adc-4de3-8c5b-37584f6ba534-tmp-dir\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:50:57.547629 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.547518 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b43b6a10-f027-4a3c-8e2a-fe42dfb285d4-cert\") pod \"ingress-canary-crv44\" (UID: \"b43b6a10-f027-4a3c-8e2a-fe42dfb285d4\") " pod="openshift-ingress-canary/ingress-canary-crv44" Sep 09 20:50:57.547629 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.547538 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phr6p\" (UniqueName: \"kubernetes.io/projected/df144005-7adc-4de3-8c5b-37584f6ba534-kube-api-access-phr6p\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:50:57.547629 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.547585 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dmp4\" (UniqueName: \"kubernetes.io/projected/b43b6a10-f027-4a3c-8e2a-fe42dfb285d4-kube-api-access-7dmp4\") pod \"ingress-canary-crv44\" (UID: \"b43b6a10-f027-4a3c-8e2a-fe42dfb285d4\") " pod="openshift-ingress-canary/ingress-canary-crv44" Sep 09 20:50:57.547768 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.547643 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:50:57.632817 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.632780 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:50:57.632996 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.632795 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:50:57.635937 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.635636 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 20:50:57.635937 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.635655 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kjjjq\"" Sep 09 20:50:57.635937 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.635661 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 20:50:57.635937 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.635751 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wskns\"" Sep 09 20:50:57.635937 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.635937 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 20:50:57.648482 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.648457 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dmp4\" (UniqueName: \"kubernetes.io/projected/b43b6a10-f027-4a3c-8e2a-fe42dfb285d4-kube-api-access-7dmp4\") pod \"ingress-canary-crv44\" (UID: \"b43b6a10-f027-4a3c-8e2a-fe42dfb285d4\") " pod="openshift-ingress-canary/ingress-canary-crv44" Sep 09 20:50:57.648599 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.648498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:50:57.648662 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:57.648607 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:50:57.648770 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:57.648667 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls podName:df144005-7adc-4de3-8c5b-37584f6ba534 nodeName:}" failed. No retries permitted until 2026-09-09 20:50:58.148645535 +0000 UTC m=+34.987844644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls") pod "dns-default-8wlxq" (UID: "df144005-7adc-4de3-8c5b-37584f6ba534") : secret "dns-default-metrics-tls" not found Sep 09 20:50:57.648945 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.648839 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/df144005-7adc-4de3-8c5b-37584f6ba534-config-volume\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:50:57.648945 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.648905 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df144005-7adc-4de3-8c5b-37584f6ba534-tmp-dir\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:50:57.648945 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.648937 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b43b6a10-f027-4a3c-8e2a-fe42dfb285d4-cert\") pod \"ingress-canary-crv44\" (UID: \"b43b6a10-f027-4a3c-8e2a-fe42dfb285d4\") " pod="openshift-ingress-canary/ingress-canary-crv44" Sep 09 20:50:57.649092 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.648966 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phr6p\" (UniqueName: \"kubernetes.io/projected/df144005-7adc-4de3-8c5b-37584f6ba534-kube-api-access-phr6p\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:50:57.649484 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.649465 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/df144005-7adc-4de3-8c5b-37584f6ba534-config-volume\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:50:57.650264 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.650198 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df144005-7adc-4de3-8c5b-37584f6ba534-tmp-dir\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:50:57.653717 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.653697 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b43b6a10-f027-4a3c-8e2a-fe42dfb285d4-cert\") pod \"ingress-canary-crv44\" (UID: \"b43b6a10-f027-4a3c-8e2a-fe42dfb285d4\") " pod="openshift-ingress-canary/ingress-canary-crv44" Sep 09 20:50:57.657294 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.657268 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phr6p\" (UniqueName: \"kubernetes.io/projected/df144005-7adc-4de3-8c5b-37584f6ba534-kube-api-access-phr6p\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:50:57.657609 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.657586 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dmp4\" (UniqueName: \"kubernetes.io/projected/b43b6a10-f027-4a3c-8e2a-fe42dfb285d4-kube-api-access-7dmp4\") pod \"ingress-canary-crv44\" (UID: \"b43b6a10-f027-4a3c-8e2a-fe42dfb285d4\") " pod="openshift-ingress-canary/ingress-canary-crv44" Sep 09 20:50:57.748535 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:57.748485 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-crv44" Sep 09 20:50:58.153581 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:58.153489 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:50:58.153736 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:58.153649 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:50:58.153736 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:58.153728 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls podName:df144005-7adc-4de3-8c5b-37584f6ba534 nodeName:}" failed. No retries permitted until 2026-09-09 20:50:59.153710592 +0000 UTC m=+35.992909701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls") pod "dns-default-8wlxq" (UID: "df144005-7adc-4de3-8c5b-37584f6ba534") : secret "dns-default-metrics-tls" not found Sep 09 20:50:58.633168 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:58.633130 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:50:58.635843 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:58.635819 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 20:50:59.002776 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:59.002614 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-crv44"] Sep 09 20:50:59.021101 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:50:59.021061 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb43b6a10_f027_4a3c_8e2a_fe42dfb285d4.slice/crio-acafc75c32d3e85de035e4f0c87e2a9b2661449e7bf96aab4997293a2164d59e WatchSource:0}: Error finding container acafc75c32d3e85de035e4f0c87e2a9b2661449e7bf96aab4997293a2164d59e: Status 404 returned error can't find the container with id acafc75c32d3e85de035e4f0c87e2a9b2661449e7bf96aab4997293a2164d59e Sep 09 20:50:59.159619 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:59.159529 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:50:59.159758 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:59.159672 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:50:59.159758 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:50:59.159733 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls podName:df144005-7adc-4de3-8c5b-37584f6ba534 nodeName:}" failed. No retries permitted until 2026-09-09 20:51:01.159718389 +0000 UTC m=+37.998917496 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls") pod "dns-default-8wlxq" (UID: "df144005-7adc-4de3-8c5b-37584f6ba534") : secret "dns-default-metrics-tls" not found Sep 09 20:50:59.844749 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:59.844717 2562 generic.go:356] "Generic (PLEG): container finished" podID="e26cba9c-5ba0-44c8-9d41-c63dad722bbe" containerID="79a72d6fad95ec83a8f8e3b991eca0b938e4b7b9598f425d039768ff6e2669c4" exitCode=0 Sep 09 20:50:59.844917 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:59.844811 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prbnq" event={"ID":"e26cba9c-5ba0-44c8-9d41-c63dad722bbe","Type":"ContainerDied","Data":"79a72d6fad95ec83a8f8e3b991eca0b938e4b7b9598f425d039768ff6e2669c4"} Sep 09 20:50:59.845946 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:50:59.845925 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-crv44" event={"ID":"b43b6a10-f027-4a3c-8e2a-fe42dfb285d4","Type":"ContainerStarted","Data":"acafc75c32d3e85de035e4f0c87e2a9b2661449e7bf96aab4997293a2164d59e"} Sep 09 20:51:00.852641 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:00.852606 2562 generic.go:356] "Generic (PLEG): container finished" podID="e26cba9c-5ba0-44c8-9d41-c63dad722bbe" containerID="6eace84ad2752baf546c44938982b52baddc0612b1d6bfea9826cbe8352ff4d9" exitCode=0 Sep 09 20:51:00.853172 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:00.852666 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prbnq" event={"ID":"e26cba9c-5ba0-44c8-9d41-c63dad722bbe","Type":"ContainerDied","Data":"6eace84ad2752baf546c44938982b52baddc0612b1d6bfea9826cbe8352ff4d9"} Sep 09 20:51:01.178027 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:01.177948 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:51:01.178197 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:51:01.178126 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:51:01.178309 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:51:01.178198 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls podName:df144005-7adc-4de3-8c5b-37584f6ba534 nodeName:}" failed. No retries permitted until 2026-09-09 20:51:05.178181546 +0000 UTC m=+42.017380655 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls") pod "dns-default-8wlxq" (UID: "df144005-7adc-4de3-8c5b-37584f6ba534") : secret "dns-default-metrics-tls" not found Sep 09 20:51:01.859531 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:01.859497 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prbnq" event={"ID":"e26cba9c-5ba0-44c8-9d41-c63dad722bbe","Type":"ContainerStarted","Data":"77d34212aa3ac1d7a44e26f9b75a93552808a1c473c8c9f3cda18e11eedeea1d"} Sep 09 20:51:01.880394 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:01.880339 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-prbnq" podStartSLOduration=10.165549164 podStartE2EDuration="38.880322938s" totalImagesPullingTime="28.714773774s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:50:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:51:01.879081686 +0000 UTC m=+38.718280817" watchObservedRunningTime="2026-09-09 20:51:01.880322938 +0000 UTC m=+38.719522045" Sep 09 20:51:02.863327 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:02.863285 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-crv44" event={"ID":"b43b6a10-f027-4a3c-8e2a-fe42dfb285d4","Type":"ContainerStarted","Data":"c4ccad19cf1ce60335f21ed21a4c81443ae41e96f9b2f61db9d92d6b26ede735"} Sep 09 20:51:02.878495 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:02.878449 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-crv44" podStartSLOduration=2.682783897 podStartE2EDuration="5.878433553s" totalImagesPullingTime="3.195649656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:50:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:51:02.87796695 +0000 UTC m=+39.717166081" watchObservedRunningTime="2026-09-09 20:51:02.878433553 +0000 UTC m=+39.717632682" Sep 09 20:51:05.207888 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:05.207839 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:51:05.208294 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:51:05.208035 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:51:05.208294 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:51:05.208107 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls podName:df144005-7adc-4de3-8c5b-37584f6ba534 nodeName:}" failed. No retries permitted until 2026-09-09 20:51:13.208090564 +0000 UTC m=+50.047289672 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls") pod "dns-default-8wlxq" (UID: "df144005-7adc-4de3-8c5b-37584f6ba534") : secret "dns-default-metrics-tls" not found Sep 09 20:51:13.268112 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:13.268067 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:51:13.268681 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:51:13.268257 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:51:13.268681 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:51:13.268336 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls podName:df144005-7adc-4de3-8c5b-37584f6ba534 nodeName:}" failed. No retries permitted until 2026-09-09 20:51:29.26831324 +0000 UTC m=+66.107512354 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls") pod "dns-default-8wlxq" (UID: "df144005-7adc-4de3-8c5b-37584f6ba534") : secret "dns-default-metrics-tls" not found Sep 09 20:51:23.832927 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:23.832890 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kjz44" Sep 09 20:51:29.275624 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:29.275577 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs\") pod \"network-metrics-daemon-lqd29\" (UID: \"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5\") " pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:51:29.276113 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:29.275666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:51:29.276113 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:51:29.275753 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:51:29.276113 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:51:29.275806 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls podName:df144005-7adc-4de3-8c5b-37584f6ba534 nodeName:}" failed. No retries permitted until 2026-09-09 20:52:01.275793818 +0000 UTC m=+98.114992939 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls") pod "dns-default-8wlxq" (UID: "df144005-7adc-4de3-8c5b-37584f6ba534") : secret "dns-default-metrics-tls" not found Sep 09 20:51:29.278028 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:29.278006 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 20:51:29.286119 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:51:29.286100 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 20:51:29.286230 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:51:29.286221 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs podName:ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5 nodeName:}" failed. No retries permitted until 2026-09-09 20:52:33.286198904 +0000 UTC m=+130.125398017 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs") pod "network-metrics-daemon-lqd29" (UID: "ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5") : secret "metrics-daemon-secret" not found Sep 09 20:51:29.376400 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:29.376360 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret\") pod \"global-pull-secret-syncer-x59k7\" (UID: \"79c3cb22-0efb-42bc-8697-ad6431a2a874\") " pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:51:29.378571 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:29.378551 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 20:51:29.390293 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:29.390264 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79c3cb22-0efb-42bc-8697-ad6431a2a874-original-pull-secret\") pod \"global-pull-secret-syncer-x59k7\" (UID: \"79c3cb22-0efb-42bc-8697-ad6431a2a874\") " pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:51:29.477070 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:29.477037 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbgnn\" (UniqueName: \"kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn\") pod \"network-check-target-vtb42\" (UID: \"f57dac14-ec36-409f-b8ef-e7a993c313ae\") " pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:51:29.479476 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:29.479459 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 20:51:29.489474 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:29.489456 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 20:51:29.500960 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:29.500926 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbgnn\" (UniqueName: \"kubernetes.io/projected/f57dac14-ec36-409f-b8ef-e7a993c313ae-kube-api-access-jbgnn\") pod \"network-check-target-vtb42\" (UID: \"f57dac14-ec36-409f-b8ef-e7a993c313ae\") " pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:51:29.543773 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:29.543706 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x59k7" Sep 09 20:51:29.747632 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:29.747596 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wskns\"" Sep 09 20:51:29.755615 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:29.755589 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:51:29.847228 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:29.847199 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x59k7"] Sep 09 20:51:29.850304 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:51:29.850280 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c3cb22_0efb_42bc_8697_ad6431a2a874.slice/crio-14c405541ccc500fa0f664fecf3b49bff125a986fb2348ecbcf49b5387d8d400 WatchSource:0}: Error finding container 14c405541ccc500fa0f664fecf3b49bff125a986fb2348ecbcf49b5387d8d400: Status 404 returned error can't find the container with id 14c405541ccc500fa0f664fecf3b49bff125a986fb2348ecbcf49b5387d8d400 Sep 09 20:51:29.927963 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:29.927932 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x59k7" event={"ID":"79c3cb22-0efb-42bc-8697-ad6431a2a874","Type":"ContainerStarted","Data":"14c405541ccc500fa0f664fecf3b49bff125a986fb2348ecbcf49b5387d8d400"} Sep 09 20:51:30.039286 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:30.039212 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vtb42"] Sep 09 20:51:30.042875 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:51:30.042841 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf57dac14_ec36_409f_b8ef_e7a993c313ae.slice/crio-899714181a3308ebee53f208719ea819b63456c41d6575f380a018834a9b9d17 WatchSource:0}: Error finding container 899714181a3308ebee53f208719ea819b63456c41d6575f380a018834a9b9d17: Status 404 returned error can't find the container with id 899714181a3308ebee53f208719ea819b63456c41d6575f380a018834a9b9d17 Sep 09 20:51:30.933852 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:30.933809 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vtb42" event={"ID":"f57dac14-ec36-409f-b8ef-e7a993c313ae","Type":"ContainerStarted","Data":"899714181a3308ebee53f208719ea819b63456c41d6575f380a018834a9b9d17"} Sep 09 20:51:34.945353 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:34.945230 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vtb42" event={"ID":"f57dac14-ec36-409f-b8ef-e7a993c313ae","Type":"ContainerStarted","Data":"23e8ca92b73f92221db0ee730c53693041a05f24599d024d03587e7dfdc402d7"} Sep 09 20:51:34.945797 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:34.945359 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:51:34.946646 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:34.946627 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x59k7" event={"ID":"79c3cb22-0efb-42bc-8697-ad6431a2a874","Type":"ContainerStarted","Data":"960fa20802935f32a0e0f6ee98616af589d30ddc6394b13e43bdf42f6dc0b221"} Sep 09 20:51:34.961018 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:34.960973 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vtb42" podStartSLOduration=67.533703085 podStartE2EDuration="1m11.960955774s" totalImagesPullingTime="4.427252689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:50:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:51:34.960363813 +0000 UTC m=+71.799562943" watchObservedRunningTime="2026-09-09 20:51:34.960955774 +0000 UTC m=+71.800154905" Sep 09 20:51:34.974167 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:34.974115 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-x59k7" podStartSLOduration=67.348283712 podStartE2EDuration="1m11.97409677s" totalImagesPullingTime="4.625813058s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:50:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:51:34.973598133 +0000 UTC m=+71.812797262" watchObservedRunningTime="2026-09-09 20:51:34.97409677 +0000 UTC m=+71.813295901" Sep 09 20:51:49.853142 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:49.849584 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f7c46849b-jll7z"] Sep 09 20:51:49.853652 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:49.853636 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f7c46849b-jll7z" Sep 09 20:51:49.856037 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:49.856015 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 09 20:51:49.857060 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:49.857038 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 09 20:51:49.857167 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:49.857040 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 09 20:51:49.857167 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:49.857044 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-xfmnc\"" Sep 09 20:51:49.857167 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:49.857136 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 09 20:51:49.863650 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:49.863629 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f7c46849b-jll7z"] Sep 09 20:51:49.913084 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:49.913045 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2shzr\" (UniqueName: \"kubernetes.io/projected/07d703dc-edf6-431b-99a8-1c5825d17712-kube-api-access-2shzr\") pod \"managed-serviceaccount-addon-agent-f7c46849b-jll7z\" (UID: \"07d703dc-edf6-431b-99a8-1c5825d17712\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f7c46849b-jll7z" Sep 09 20:51:49.913281 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:49.913112 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07d703dc-edf6-431b-99a8-1c5825d17712-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-f7c46849b-jll7z\" (UID: \"07d703dc-edf6-431b-99a8-1c5825d17712\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f7c46849b-jll7z" Sep 09 20:51:50.013411 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:50.013354 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2shzr\" (UniqueName: \"kubernetes.io/projected/07d703dc-edf6-431b-99a8-1c5825d17712-kube-api-access-2shzr\") pod \"managed-serviceaccount-addon-agent-f7c46849b-jll7z\" (UID: \"07d703dc-edf6-431b-99a8-1c5825d17712\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f7c46849b-jll7z" Sep 09 20:51:50.013591 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:50.013440 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07d703dc-edf6-431b-99a8-1c5825d17712-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-f7c46849b-jll7z\" (UID: \"07d703dc-edf6-431b-99a8-1c5825d17712\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f7c46849b-jll7z" Sep 09 20:51:50.016059 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:50.016038 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07d703dc-edf6-431b-99a8-1c5825d17712-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-f7c46849b-jll7z\" (UID: \"07d703dc-edf6-431b-99a8-1c5825d17712\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f7c46849b-jll7z" Sep 09 20:51:50.022278 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:50.022254 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2shzr\" (UniqueName: \"kubernetes.io/projected/07d703dc-edf6-431b-99a8-1c5825d17712-kube-api-access-2shzr\") pod \"managed-serviceaccount-addon-agent-f7c46849b-jll7z\" (UID: \"07d703dc-edf6-431b-99a8-1c5825d17712\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f7c46849b-jll7z" Sep 09 20:51:50.172735 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:50.172654 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f7c46849b-jll7z" Sep 09 20:51:50.437610 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:50.437535 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f7c46849b-jll7z"] Sep 09 20:51:50.440037 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:51:50.440010 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07d703dc_edf6_431b_99a8_1c5825d17712.slice/crio-fff6d55083a3a6cd8c27249c970ddff6e671f0e6010adad51e5a1644b903621b WatchSource:0}: Error finding container fff6d55083a3a6cd8c27249c970ddff6e671f0e6010adad51e5a1644b903621b: Status 404 returned error can't find the container with id fff6d55083a3a6cd8c27249c970ddff6e671f0e6010adad51e5a1644b903621b Sep 09 20:51:50.985336 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:50.985298 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f7c46849b-jll7z" event={"ID":"07d703dc-edf6-431b-99a8-1c5825d17712","Type":"ContainerStarted","Data":"fff6d55083a3a6cd8c27249c970ddff6e671f0e6010adad51e5a1644b903621b"} Sep 09 20:51:53.994829 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:53.994788 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f7c46849b-jll7z" event={"ID":"07d703dc-edf6-431b-99a8-1c5825d17712","Type":"ContainerStarted","Data":"0364e5b4bfd34b5dca49cc367f5c5478d88389363e39d50b6b5a4c0d5277380f"} Sep 09 20:51:54.010735 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:51:54.010683 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f7c46849b-jll7z" podStartSLOduration=2.17193964 podStartE2EDuration="5.010666294s" totalImagesPullingTime="2.838726654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:51:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:51:54.010098306 +0000 UTC m=+90.849297437" watchObservedRunningTime="2026-09-09 20:51:54.010666294 +0000 UTC m=+90.849865423" Sep 09 20:52:01.292953 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:52:01.292896 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:52:01.293457 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:52:01.293021 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:52:01.293457 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:52:01.293074 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls podName:df144005-7adc-4de3-8c5b-37584f6ba534 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:05.293061669 +0000 UTC m=+162.132260777 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls") pod "dns-default-8wlxq" (UID: "df144005-7adc-4de3-8c5b-37584f6ba534") : secret "dns-default-metrics-tls" not found Sep 09 20:52:05.951862 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:52:05.951822 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vtb42" Sep 09 20:52:33.305346 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:52:33.305292 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs\") pod \"network-metrics-daemon-lqd29\" (UID: \"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5\") " pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:52:33.305849 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:52:33.305446 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 20:52:33.305849 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:52:33.305520 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs podName:ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5 nodeName:}" failed. No retries permitted until 2026-09-09 20:54:35.305502996 +0000 UTC m=+252.144702103 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs") pod "network-metrics-daemon-lqd29" (UID: "ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5") : secret "metrics-daemon-secret" not found Sep 09 20:53:00.443085 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:53:00.443044 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8wlxq" podUID="df144005-7adc-4de3-8c5b-37584f6ba534" Sep 09 20:53:00.652344 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:53:00.652279 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:53:01.157037 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:53:01.157007 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8wlxq" Sep 09 20:53:05.314489 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:53:05.314450 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:53:05.314963 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:53:05.314596 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 20:53:05.314963 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:53:05.314663 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls podName:df144005-7adc-4de3-8c5b-37584f6ba534 nodeName:}" failed. No retries permitted until 2026-09-09 20:55:07.314647385 +0000 UTC m=+284.153846493 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls") pod "dns-default-8wlxq" (UID: "df144005-7adc-4de3-8c5b-37584f6ba534") : secret "dns-default-metrics-tls" not found Sep 09 20:53:14.632943 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:53:14.632853 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:53:54.281883 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:53:54.281849 2562 generic.go:356] "Generic (PLEG): container finished" podID="07d703dc-edf6-431b-99a8-1c5825d17712" containerID="0364e5b4bfd34b5dca49cc367f5c5478d88389363e39d50b6b5a4c0d5277380f" exitCode=255 Sep 09 20:53:54.282313 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:53:54.281921 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f7c46849b-jll7z" event={"ID":"07d703dc-edf6-431b-99a8-1c5825d17712","Type":"ContainerDied","Data":"0364e5b4bfd34b5dca49cc367f5c5478d88389363e39d50b6b5a4c0d5277380f"} Sep 09 20:53:54.287311 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:53:54.287295 2562 scope.go:117] "RemoveContainer" containerID="0364e5b4bfd34b5dca49cc367f5c5478d88389363e39d50b6b5a4c0d5277380f" Sep 09 20:53:55.286198 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:53:55.286165 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f7c46849b-jll7z" event={"ID":"07d703dc-edf6-431b-99a8-1c5825d17712","Type":"ContainerStarted","Data":"3f6b67c56114182abf252eba3ca583af80dd0d43de3f0c5ad685559e8e656789"} Sep 09 20:54:35.325411 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:35.325353 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs\") pod \"network-metrics-daemon-lqd29\" (UID: \"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5\") " pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:54:35.325916 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:35.325511 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 20:54:35.325916 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:35.325581 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs podName:ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5 nodeName:}" failed. No retries permitted until 2026-09-09 20:56:37.325563665 +0000 UTC m=+374.164762773 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs") pod "network-metrics-daemon-lqd29" (UID: "ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5") : secret "metrics-daemon-secret" not found Sep 09 20:54:43.869481 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.869446 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78888988f7-jf6jb"] Sep 09 20:54:43.872230 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.872214 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:43.875963 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.875929 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 09 20:54:43.876402 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.876384 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 09 20:54:43.876973 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.876955 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dpwvf\"" Sep 09 20:54:43.878758 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.878740 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 09 20:54:43.881981 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.881963 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 09 20:54:43.899086 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.899057 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78888988f7-jf6jb"] Sep 09 20:54:43.970499 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.970466 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s"] Sep 09 20:54:43.973265 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.973228 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx"] Sep 09 20:54:43.973395 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.973338 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s" Sep 09 20:54:43.976225 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.976193 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-2mxsn"] Sep 09 20:54:43.976369 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.976291 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 09 20:54:43.976369 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.976321 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" Sep 09 20:54:43.976537 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.976518 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-7jpxp\"" Sep 09 20:54:43.977175 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.977154 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 09 20:54:43.978406 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.978386 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 09 20:54:43.978799 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.978779 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 09 20:54:43.979179 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.979163 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 09 20:54:43.979345 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.979325 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-57twl\"" Sep 09 20:54:43.979630 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.979616 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 09 20:54:43.979868 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.979635 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 09 20:54:43.980040 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.980018 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:43.983258 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.983222 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 09 20:54:43.983447 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.983380 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-c2p5t\"" Sep 09 20:54:43.983626 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.983551 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 09 20:54:43.983884 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.983785 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 09 20:54:43.983884 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.983845 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 09 20:54:43.984049 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.983989 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzwgh\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-kube-api-access-dzwgh\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:43.984049 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.984020 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/372b1efb-f45d-4dae-9d3a-2070ad8099b2-installation-pull-secrets\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:43.984162 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.984052 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/372b1efb-f45d-4dae-9d3a-2070ad8099b2-ca-trust-extracted\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:43.984162 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.984105 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:43.984345 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.984162 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-certificates\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:43.984345 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.984216 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/372b1efb-f45d-4dae-9d3a-2070ad8099b2-image-registry-private-configuration\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:43.984345 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.984273 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s"] Sep 09 20:54:43.984345 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.984282 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/372b1efb-f45d-4dae-9d3a-2070ad8099b2-trusted-ca\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:43.984345 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.984342 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-bound-sa-token\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:43.988922 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.988869 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx"] Sep 09 20:54:43.989757 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.989739 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 09 20:54:43.998323 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:43.998300 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-2mxsn"] Sep 09 20:54:44.084818 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.084790 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/372b1efb-f45d-4dae-9d3a-2070ad8099b2-ca-trust-extracted\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:44.084818 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.084823 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5879ea93-d3c6-4607-a910-536119156bcf-tmp\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.085071 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.084843 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c9eaaaa8-54b7-4c98-b4a6-643a2826f899-config\") pod \"service-ca-operator-859f94c4b-r829s\" (UID: \"c9eaaaa8-54b7-4c98-b4a6-643a2826f899\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s" Sep 09 20:54:44.085071 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.084969 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:44.085071 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.084988 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-certificates\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:44.085071 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.085009 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9eaaaa8-54b7-4c98-b4a6-643a2826f899-serving-cert\") pod \"service-ca-operator-859f94c4b-r829s\" (UID: \"c9eaaaa8-54b7-4c98-b4a6-643a2826f899\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s" Sep 09 20:54:44.085071 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.085032 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhh8k\" (UniqueName: \"kubernetes.io/projected/14a43885-6f91-41fc-bafe-08fec415b79e-kube-api-access-rhh8k\") pod \"cluster-samples-operator-7cdcfcf76c-dvhjx\" (UID: \"14a43885-6f91-41fc-bafe-08fec415b79e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" Sep 09 20:54:44.085344 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:44.085094 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 20:54:44.085344 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:44.085106 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78888988f7-jf6jb: secret "image-registry-tls" not found Sep 09 20:54:44.085344 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:44.085159 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls podName:372b1efb-f45d-4dae-9d3a-2070ad8099b2 nodeName:}" failed. No retries permitted until 2026-09-09 20:54:44.585145502 +0000 UTC m=+261.424344609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls") pod "image-registry-78888988f7-jf6jb" (UID: "372b1efb-f45d-4dae-9d3a-2070ad8099b2") : secret "image-registry-tls" not found Sep 09 20:54:44.085344 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.085230 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/372b1efb-f45d-4dae-9d3a-2070ad8099b2-image-registry-private-configuration\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:44.085344 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.085284 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/372b1efb-f45d-4dae-9d3a-2070ad8099b2-trusted-ca\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:44.085344 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.085318 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-dvhjx\" (UID: \"14a43885-6f91-41fc-bafe-08fec415b79e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" Sep 09 20:54:44.085344 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.085228 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/372b1efb-f45d-4dae-9d3a-2070ad8099b2-ca-trust-extracted\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:44.085673 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.085358 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-bound-sa-token\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:44.085673 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.085383 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5879ea93-d3c6-4607-a910-536119156bcf-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.085673 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.085407 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5879ea93-d3c6-4607-a910-536119156bcf-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.085673 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.085433 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf8qv\" (UniqueName: \"kubernetes.io/projected/5879ea93-d3c6-4607-a910-536119156bcf-kube-api-access-hf8qv\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.085673 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.085497 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzwgh\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-kube-api-access-dzwgh\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:44.085673 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.085557 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-certificates\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:44.085673 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.085595 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/372b1efb-f45d-4dae-9d3a-2070ad8099b2-installation-pull-secrets\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:44.085673 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.085629 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzzz2\" (UniqueName: \"kubernetes.io/projected/c9eaaaa8-54b7-4c98-b4a6-643a2826f899-kube-api-access-fzzz2\") pod \"service-ca-operator-859f94c4b-r829s\" (UID: \"c9eaaaa8-54b7-4c98-b4a6-643a2826f899\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s" Sep 09 20:54:44.085673 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.085660 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5879ea93-d3c6-4607-a910-536119156bcf-snapshots\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.085976 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.085686 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5879ea93-d3c6-4607-a910-536119156bcf-serving-cert\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.086106 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.086089 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/372b1efb-f45d-4dae-9d3a-2070ad8099b2-trusted-ca\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:44.087744 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.087720 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/372b1efb-f45d-4dae-9d3a-2070ad8099b2-image-registry-private-configuration\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:44.087837 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.087811 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/372b1efb-f45d-4dae-9d3a-2070ad8099b2-installation-pull-secrets\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:44.096860 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.096835 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-bound-sa-token\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:44.097061 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.097039 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzwgh\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-kube-api-access-dzwgh\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:44.186352 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.186258 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5879ea93-d3c6-4607-a910-536119156bcf-snapshots\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.186352 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.186295 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5879ea93-d3c6-4607-a910-536119156bcf-serving-cert\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.186352 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.186317 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5879ea93-d3c6-4607-a910-536119156bcf-tmp\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.186630 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.186388 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c9eaaaa8-54b7-4c98-b4a6-643a2826f899-config\") pod \"service-ca-operator-859f94c4b-r829s\" (UID: \"c9eaaaa8-54b7-4c98-b4a6-643a2826f899\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s" Sep 09 20:54:44.186630 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.186448 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9eaaaa8-54b7-4c98-b4a6-643a2826f899-serving-cert\") pod \"service-ca-operator-859f94c4b-r829s\" (UID: \"c9eaaaa8-54b7-4c98-b4a6-643a2826f899\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s" Sep 09 20:54:44.186630 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.186471 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhh8k\" (UniqueName: \"kubernetes.io/projected/14a43885-6f91-41fc-bafe-08fec415b79e-kube-api-access-rhh8k\") pod \"cluster-samples-operator-7cdcfcf76c-dvhjx\" (UID: \"14a43885-6f91-41fc-bafe-08fec415b79e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" Sep 09 20:54:44.186630 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.186508 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-dvhjx\" (UID: \"14a43885-6f91-41fc-bafe-08fec415b79e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" Sep 09 20:54:44.186630 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.186540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5879ea93-d3c6-4607-a910-536119156bcf-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.186881 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:44.186841 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 20:54:44.186930 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:44.186916 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls podName:14a43885-6f91-41fc-bafe-08fec415b79e nodeName:}" failed. No retries permitted until 2026-09-09 20:54:44.686895995 +0000 UTC m=+261.526095104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-dvhjx" (UID: "14a43885-6f91-41fc-bafe-08fec415b79e") : secret "samples-operator-tls" not found Sep 09 20:54:44.186990 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.186973 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5879ea93-d3c6-4607-a910-536119156bcf-snapshots\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.187046 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.186975 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5879ea93-d3c6-4607-a910-536119156bcf-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.187046 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.187037 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf8qv\" (UniqueName: \"kubernetes.io/projected/5879ea93-d3c6-4607-a910-536119156bcf-kube-api-access-hf8qv\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.187329 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.187061 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c9eaaaa8-54b7-4c98-b4a6-643a2826f899-config\") pod \"service-ca-operator-859f94c4b-r829s\" (UID: \"c9eaaaa8-54b7-4c98-b4a6-643a2826f899\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s" Sep 09 20:54:44.187329 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.187093 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzzz2\" (UniqueName: \"kubernetes.io/projected/c9eaaaa8-54b7-4c98-b4a6-643a2826f899-kube-api-access-fzzz2\") pod \"service-ca-operator-859f94c4b-r829s\" (UID: \"c9eaaaa8-54b7-4c98-b4a6-643a2826f899\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s" Sep 09 20:54:44.187329 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.187313 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5879ea93-d3c6-4607-a910-536119156bcf-tmp\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.187613 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.187590 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5879ea93-d3c6-4607-a910-536119156bcf-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.187677 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.187632 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5879ea93-d3c6-4607-a910-536119156bcf-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.188879 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.188857 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9eaaaa8-54b7-4c98-b4a6-643a2826f899-serving-cert\") pod \"service-ca-operator-859f94c4b-r829s\" (UID: \"c9eaaaa8-54b7-4c98-b4a6-643a2826f899\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s" Sep 09 20:54:44.188988 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.188971 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5879ea93-d3c6-4607-a910-536119156bcf-serving-cert\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.195470 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.195437 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzzz2\" (UniqueName: \"kubernetes.io/projected/c9eaaaa8-54b7-4c98-b4a6-643a2826f899-kube-api-access-fzzz2\") pod \"service-ca-operator-859f94c4b-r829s\" (UID: \"c9eaaaa8-54b7-4c98-b4a6-643a2826f899\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s" Sep 09 20:54:44.195621 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.195570 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhh8k\" (UniqueName: \"kubernetes.io/projected/14a43885-6f91-41fc-bafe-08fec415b79e-kube-api-access-rhh8k\") pod \"cluster-samples-operator-7cdcfcf76c-dvhjx\" (UID: \"14a43885-6f91-41fc-bafe-08fec415b79e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" Sep 09 20:54:44.195831 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.195810 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf8qv\" (UniqueName: \"kubernetes.io/projected/5879ea93-d3c6-4607-a910-536119156bcf-kube-api-access-hf8qv\") pod \"insights-operator-76bcd6ccf5-2mxsn\" (UID: \"5879ea93-d3c6-4607-a910-536119156bcf\") " pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.287371 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.287336 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s" Sep 09 20:54:44.299872 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.299851 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" Sep 09 20:54:44.562781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.562701 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s"] Sep 09 20:54:44.565369 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:54:44.565344 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9eaaaa8_54b7_4c98_b4a6_643a2826f899.slice/crio-e78be4eddb14f7884764ee963b2e9beaf6d08e17a90902d3666164e2f7245a08 WatchSource:0}: Error finding container e78be4eddb14f7884764ee963b2e9beaf6d08e17a90902d3666164e2f7245a08: Status 404 returned error can't find the container with id e78be4eddb14f7884764ee963b2e9beaf6d08e17a90902d3666164e2f7245a08 Sep 09 20:54:44.578691 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.578661 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-2mxsn"] Sep 09 20:54:44.581533 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:54:44.581510 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5879ea93_d3c6_4607_a910_536119156bcf.slice/crio-b356310bd3475334517de761a9c375bdecc9a37deca9b9e44c26f528bee7c54a WatchSource:0}: Error finding container b356310bd3475334517de761a9c375bdecc9a37deca9b9e44c26f528bee7c54a: Status 404 returned error can't find the container with id b356310bd3475334517de761a9c375bdecc9a37deca9b9e44c26f528bee7c54a Sep 09 20:54:44.589123 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.589101 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:44.589267 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:44.589252 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 20:54:44.589318 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:44.589269 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78888988f7-jf6jb: secret "image-registry-tls" not found Sep 09 20:54:44.589318 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:44.589316 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls podName:372b1efb-f45d-4dae-9d3a-2070ad8099b2 nodeName:}" failed. No retries permitted until 2026-09-09 20:54:45.589299073 +0000 UTC m=+262.428498201 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls") pod "image-registry-78888988f7-jf6jb" (UID: "372b1efb-f45d-4dae-9d3a-2070ad8099b2") : secret "image-registry-tls" not found Sep 09 20:54:44.690431 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:44.690340 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-dvhjx\" (UID: \"14a43885-6f91-41fc-bafe-08fec415b79e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" Sep 09 20:54:44.690585 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:44.690513 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 20:54:44.690669 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:44.690592 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls podName:14a43885-6f91-41fc-bafe-08fec415b79e nodeName:}" failed. No retries permitted until 2026-09-09 20:54:45.690569783 +0000 UTC m=+262.529768908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-dvhjx" (UID: "14a43885-6f91-41fc-bafe-08fec415b79e") : secret "samples-operator-tls" not found Sep 09 20:54:45.411198 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:45.411162 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s" event={"ID":"c9eaaaa8-54b7-4c98-b4a6-643a2826f899","Type":"ContainerStarted","Data":"e78be4eddb14f7884764ee963b2e9beaf6d08e17a90902d3666164e2f7245a08"} Sep 09 20:54:45.412680 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:45.412651 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" event={"ID":"5879ea93-d3c6-4607-a910-536119156bcf","Type":"ContainerStarted","Data":"b356310bd3475334517de761a9c375bdecc9a37deca9b9e44c26f528bee7c54a"} Sep 09 20:54:45.597512 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:45.597470 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:45.597687 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:45.597626 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 20:54:45.597687 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:45.597645 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78888988f7-jf6jb: secret "image-registry-tls" not found Sep 09 20:54:45.597793 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:45.597697 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls podName:372b1efb-f45d-4dae-9d3a-2070ad8099b2 nodeName:}" failed. No retries permitted until 2026-09-09 20:54:47.597681685 +0000 UTC m=+264.436880792 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls") pod "image-registry-78888988f7-jf6jb" (UID: "372b1efb-f45d-4dae-9d3a-2070ad8099b2") : secret "image-registry-tls" not found Sep 09 20:54:45.698138 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:45.698052 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-dvhjx\" (UID: \"14a43885-6f91-41fc-bafe-08fec415b79e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" Sep 09 20:54:45.698326 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:45.698171 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 20:54:45.698326 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:45.698229 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls podName:14a43885-6f91-41fc-bafe-08fec415b79e nodeName:}" failed. No retries permitted until 2026-09-09 20:54:47.69821326 +0000 UTC m=+264.537412374 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-dvhjx" (UID: "14a43885-6f91-41fc-bafe-08fec415b79e") : secret "samples-operator-tls" not found Sep 09 20:54:47.422050 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:47.422008 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s" event={"ID":"c9eaaaa8-54b7-4c98-b4a6-643a2826f899","Type":"ContainerStarted","Data":"2537a53f3c1c3317ba774ddd938cb817400e1073038af95925c86910ecb119cf"} Sep 09 20:54:47.423610 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:47.423579 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" event={"ID":"5879ea93-d3c6-4607-a910-536119156bcf","Type":"ContainerStarted","Data":"6d4f1de56066d24674b43128ab703d7fc9dddb12493175a7fb83df3cc53f1056"} Sep 09 20:54:47.439907 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:47.439862 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s" podStartSLOduration=2.143039219 podStartE2EDuration="4.439849278s" totalImagesPullingTime="2.296810059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:54:47.439411823 +0000 UTC m=+264.278610949" watchObservedRunningTime="2026-09-09 20:54:47.439849278 +0000 UTC m=+264.279048408" Sep 09 20:54:47.455922 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:47.455863 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" podStartSLOduration=2.1787910249999998 podStartE2EDuration="4.455842297s" totalImagesPullingTime="2.277051272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:54:47.455745407 +0000 UTC m=+264.294944534" watchObservedRunningTime="2026-09-09 20:54:47.455842297 +0000 UTC m=+264.295041427" Sep 09 20:54:47.612328 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:47.612284 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:47.612504 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:47.612442 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 20:54:47.612504 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:47.612461 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78888988f7-jf6jb: secret "image-registry-tls" not found Sep 09 20:54:47.612587 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:47.612530 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls podName:372b1efb-f45d-4dae-9d3a-2070ad8099b2 nodeName:}" failed. No retries permitted until 2026-09-09 20:54:51.612515228 +0000 UTC m=+268.451714340 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls") pod "image-registry-78888988f7-jf6jb" (UID: "372b1efb-f45d-4dae-9d3a-2070ad8099b2") : secret "image-registry-tls" not found Sep 09 20:54:47.713180 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:47.713077 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-dvhjx\" (UID: \"14a43885-6f91-41fc-bafe-08fec415b79e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" Sep 09 20:54:47.713377 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:47.713259 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 20:54:47.713377 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:47.713335 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls podName:14a43885-6f91-41fc-bafe-08fec415b79e nodeName:}" failed. No retries permitted until 2026-09-09 20:54:51.713315877 +0000 UTC m=+268.552514989 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-dvhjx" (UID: "14a43885-6f91-41fc-bafe-08fec415b79e") : secret "samples-operator-tls" not found Sep 09 20:54:48.504290 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.504259 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vtlkg"] Sep 09 20:54:48.507555 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.507536 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vtlkg" Sep 09 20:54:48.510202 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.510145 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 09 20:54:48.510504 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.510485 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-c52dh\"" Sep 09 20:54:48.511117 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.511097 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 09 20:54:48.525788 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.525760 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vtlkg"] Sep 09 20:54:48.615593 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.615559 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dct65"] Sep 09 20:54:48.618741 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.618719 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:48.621020 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.620998 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm5bd\" (UniqueName: \"kubernetes.io/projected/15edbcc0-0649-4e9f-be70-1dcb3f2e0c61-kube-api-access-gm5bd\") pod \"migrator-fdbb8fdff-vtlkg\" (UID: \"15edbcc0-0649-4e9f-be70-1dcb3f2e0c61\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vtlkg" Sep 09 20:54:48.624254 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.624215 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 09 20:54:48.624819 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.624763 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 09 20:54:48.625178 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.625155 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9gf4v\"" Sep 09 20:54:48.634976 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.634956 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dct65"] Sep 09 20:54:48.721553 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.721521 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42671ab6-cd6a-4a2d-8494-828a140848f6-data-volume\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:48.721733 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.721562 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:48.721733 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.721580 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42671ab6-cd6a-4a2d-8494-828a140848f6-crio-socket\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:48.721733 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.721600 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8b7c\" (UniqueName: \"kubernetes.io/projected/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-api-access-p8b7c\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:48.721733 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.721651 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm5bd\" (UniqueName: \"kubernetes.io/projected/15edbcc0-0649-4e9f-be70-1dcb3f2e0c61-kube-api-access-gm5bd\") pod \"migrator-fdbb8fdff-vtlkg\" (UID: \"15edbcc0-0649-4e9f-be70-1dcb3f2e0c61\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vtlkg" Sep 09 20:54:48.721884 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.721861 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:48.748149 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.748118 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm5bd\" (UniqueName: \"kubernetes.io/projected/15edbcc0-0649-4e9f-be70-1dcb3f2e0c61-kube-api-access-gm5bd\") pod \"migrator-fdbb8fdff-vtlkg\" (UID: \"15edbcc0-0649-4e9f-be70-1dcb3f2e0c61\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vtlkg" Sep 09 20:54:48.818329 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.818220 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vtlkg" Sep 09 20:54:48.822551 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.822522 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42671ab6-cd6a-4a2d-8494-828a140848f6-data-volume\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:48.822641 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.822582 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:48.822641 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.822614 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42671ab6-cd6a-4a2d-8494-828a140848f6-crio-socket\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:48.822762 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.822647 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8b7c\" (UniqueName: \"kubernetes.io/projected/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-api-access-p8b7c\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:48.822762 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.822693 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:48.822762 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:48.822728 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 20:54:48.822905 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:48.822816 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm podName:42671ab6-cd6a-4a2d-8494-828a140848f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:54:49.322794656 +0000 UTC m=+266.161993782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dct65" (UID: "42671ab6-cd6a-4a2d-8494-828a140848f6") : configmap "kube-rbac-proxy" not found Sep 09 20:54:48.822905 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:48.822823 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 20:54:48.822905 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.822852 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42671ab6-cd6a-4a2d-8494-828a140848f6-data-volume\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:48.823053 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:48.822885 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls podName:42671ab6-cd6a-4a2d-8494-828a140848f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:54:49.322868326 +0000 UTC m=+266.162067434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dct65" (UID: "42671ab6-cd6a-4a2d-8494-828a140848f6") : secret "insights-runtime-extractor-tls" not found Sep 09 20:54:48.823053 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.822924 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42671ab6-cd6a-4a2d-8494-828a140848f6-crio-socket\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:48.834511 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:48.834488 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8b7c\" (UniqueName: \"kubernetes.io/projected/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-api-access-p8b7c\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:49.087360 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:49.087333 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vtlkg"] Sep 09 20:54:49.089581 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:54:49.089551 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15edbcc0_0649_4e9f_be70_1dcb3f2e0c61.slice/crio-865f2ad14ef0aa0a823f1383b53bd4e0ddd4d7cbf2ff2541155981d361564c01 WatchSource:0}: Error finding container 865f2ad14ef0aa0a823f1383b53bd4e0ddd4d7cbf2ff2541155981d361564c01: Status 404 returned error can't find the container with id 865f2ad14ef0aa0a823f1383b53bd4e0ddd4d7cbf2ff2541155981d361564c01 Sep 09 20:54:49.326654 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:49.326626 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:49.326805 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:49.326699 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:49.326805 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:49.326770 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 20:54:49.326805 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:49.326785 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 20:54:49.326924 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:49.326829 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls podName:42671ab6-cd6a-4a2d-8494-828a140848f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:54:50.326814703 +0000 UTC m=+267.166013816 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dct65" (UID: "42671ab6-cd6a-4a2d-8494-828a140848f6") : secret "insights-runtime-extractor-tls" not found Sep 09 20:54:49.326924 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:49.326844 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm podName:42671ab6-cd6a-4a2d-8494-828a140848f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:54:50.326836984 +0000 UTC m=+267.166036092 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dct65" (UID: "42671ab6-cd6a-4a2d-8494-828a140848f6") : configmap "kube-rbac-proxy" not found Sep 09 20:54:49.435130 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:49.435039 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vtlkg" event={"ID":"15edbcc0-0649-4e9f-be70-1dcb3f2e0c61","Type":"ContainerStarted","Data":"865f2ad14ef0aa0a823f1383b53bd4e0ddd4d7cbf2ff2541155981d361564c01"} Sep 09 20:54:50.336309 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:50.336272 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:50.336691 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:50.336358 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:50.336691 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:50.336431 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 20:54:50.336691 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:50.336436 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 20:54:50.336691 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:50.336484 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm podName:42671ab6-cd6a-4a2d-8494-828a140848f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:54:52.33647122 +0000 UTC m=+269.175670328 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dct65" (UID: "42671ab6-cd6a-4a2d-8494-828a140848f6") : configmap "kube-rbac-proxy" not found Sep 09 20:54:50.336691 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:50.336497 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls podName:42671ab6-cd6a-4a2d-8494-828a140848f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:54:52.336491235 +0000 UTC m=+269.175690343 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dct65" (UID: "42671ab6-cd6a-4a2d-8494-828a140848f6") : secret "insights-runtime-extractor-tls" not found Sep 09 20:54:50.439990 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:50.439897 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vtlkg" event={"ID":"15edbcc0-0649-4e9f-be70-1dcb3f2e0c61","Type":"ContainerStarted","Data":"59a85f68245e929be2196a3891b95c4975b8ebc4d4d7e1f16a5573f4ade427a3"} Sep 09 20:54:50.439990 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:50.439955 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vtlkg" event={"ID":"15edbcc0-0649-4e9f-be70-1dcb3f2e0c61","Type":"ContainerStarted","Data":"4184a075a7415d32d97baffba913854aba86080e44e95be63797e17e2560781a"} Sep 09 20:54:50.456753 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:50.456707 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vtlkg" podStartSLOduration=1.423633917 podStartE2EDuration="2.456693289s" totalImagesPullingTime="1.033059372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:54:50.455139108 +0000 UTC m=+267.294338237" watchObservedRunningTime="2026-09-09 20:54:50.456693289 +0000 UTC m=+267.295892419" Sep 09 20:54:51.530107 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:51.530080 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mvftd_d4d297c3-08db-42da-8621-b9b3d6ff8b9f/dns-node-resolver/0.log" Sep 09 20:54:51.645533 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:51.645505 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:51.645689 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:51.645628 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 20:54:51.645689 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:51.645640 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78888988f7-jf6jb: secret "image-registry-tls" not found Sep 09 20:54:51.645689 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:51.645685 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls podName:372b1efb-f45d-4dae-9d3a-2070ad8099b2 nodeName:}" failed. No retries permitted until 2026-09-09 20:54:59.645672628 +0000 UTC m=+276.484871736 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls") pod "image-registry-78888988f7-jf6jb" (UID: "372b1efb-f45d-4dae-9d3a-2070ad8099b2") : secret "image-registry-tls" not found Sep 09 20:54:51.746760 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:51.746722 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-dvhjx\" (UID: \"14a43885-6f91-41fc-bafe-08fec415b79e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" Sep 09 20:54:51.746928 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:51.746874 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 20:54:51.746975 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:51.746946 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls podName:14a43885-6f91-41fc-bafe-08fec415b79e nodeName:}" failed. No retries permitted until 2026-09-09 20:54:59.7469274 +0000 UTC m=+276.586126529 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-dvhjx" (UID: "14a43885-6f91-41fc-bafe-08fec415b79e") : secret "samples-operator-tls" not found Sep 09 20:54:52.351468 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:52.351426 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:52.351653 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:52.351485 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:52.351653 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:52.351557 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 20:54:52.351653 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:52.351601 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 20:54:52.351653 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:52.351631 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm podName:42671ab6-cd6a-4a2d-8494-828a140848f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:54:56.351614698 +0000 UTC m=+273.190813808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dct65" (UID: "42671ab6-cd6a-4a2d-8494-828a140848f6") : configmap "kube-rbac-proxy" not found Sep 09 20:54:52.351653 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:52.351646 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls podName:42671ab6-cd6a-4a2d-8494-828a140848f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:54:56.351640412 +0000 UTC m=+273.190839520 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dct65" (UID: "42671ab6-cd6a-4a2d-8494-828a140848f6") : secret "insights-runtime-extractor-tls" not found Sep 09 20:54:52.530411 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:52.530385 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sgvv4_51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93/node-ca/0.log" Sep 09 20:54:53.129836 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:53.129799 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-crv44_b43b6a10-f027-4a3c-8e2a-fe42dfb285d4/serve-healthcheck-canary/0.log" Sep 09 20:54:53.531036 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:53.531008 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-vtlkg_15edbcc0-0649-4e9f-be70-1dcb3f2e0c61/migrator/0.log" Sep 09 20:54:53.730318 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:53.730287 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-vtlkg_15edbcc0-0649-4e9f-be70-1dcb3f2e0c61/graceful-termination/0.log" Sep 09 20:54:56.384316 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:56.384278 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:56.384724 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:56.384335 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:54:56.384724 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:56.384412 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 20:54:56.384724 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:56.384476 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm podName:42671ab6-cd6a-4a2d-8494-828a140848f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:55:04.384457362 +0000 UTC m=+281.223656470 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dct65" (UID: "42671ab6-cd6a-4a2d-8494-828a140848f6") : configmap "kube-rbac-proxy" not found Sep 09 20:54:56.384724 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:56.384497 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 20:54:56.384724 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:54:56.384548 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls podName:42671ab6-cd6a-4a2d-8494-828a140848f6 nodeName:}" failed. No retries permitted until 2026-09-09 20:55:04.384535482 +0000 UTC m=+281.223734591 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dct65" (UID: "42671ab6-cd6a-4a2d-8494-828a140848f6") : secret "insights-runtime-extractor-tls" not found Sep 09 20:54:59.710911 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:59.710874 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:59.713528 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:59.713503 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls\") pod \"image-registry-78888988f7-jf6jb\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:59.781132 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:59.781097 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:54:59.811564 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:59.811526 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-dvhjx\" (UID: \"14a43885-6f91-41fc-bafe-08fec415b79e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" Sep 09 20:54:59.814016 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:59.813990 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/14a43885-6f91-41fc-bafe-08fec415b79e-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-dvhjx\" (UID: \"14a43885-6f91-41fc-bafe-08fec415b79e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" Sep 09 20:54:59.894229 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:54:59.894198 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" Sep 09 20:55:00.052111 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:00.052089 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78888988f7-jf6jb"] Sep 09 20:55:00.054318 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:55:00.054295 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod372b1efb_f45d_4dae_9d3a_2070ad8099b2.slice/crio-a0562858f386f7562037d870d4c68439d8b68767222f7547a3e826876c706d3b WatchSource:0}: Error finding container a0562858f386f7562037d870d4c68439d8b68767222f7547a3e826876c706d3b: Status 404 returned error can't find the container with id a0562858f386f7562037d870d4c68439d8b68767222f7547a3e826876c706d3b Sep 09 20:55:00.164429 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:00.164393 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx"] Sep 09 20:55:00.472542 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:00.472507 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78888988f7-jf6jb" event={"ID":"372b1efb-f45d-4dae-9d3a-2070ad8099b2","Type":"ContainerStarted","Data":"559f265396d09fc9bee7e0276938f63fb72ef8ec24fb6b5f1b10187f63b23559"} Sep 09 20:55:00.472716 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:00.472568 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78888988f7-jf6jb" event={"ID":"372b1efb-f45d-4dae-9d3a-2070ad8099b2","Type":"ContainerStarted","Data":"a0562858f386f7562037d870d4c68439d8b68767222f7547a3e826876c706d3b"} Sep 09 20:55:00.472716 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:00.472598 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:55:00.473720 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:00.473696 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" event={"ID":"14a43885-6f91-41fc-bafe-08fec415b79e","Type":"ContainerStarted","Data":"d1539c81bae7d319aaa659ac04c570fec7667231c7c3b94494f6193367a7463f"} Sep 09 20:55:02.482273 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:02.482223 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" event={"ID":"14a43885-6f91-41fc-bafe-08fec415b79e","Type":"ContainerStarted","Data":"189dadc939be7334b8a040c64761a513ac4111eea33f873ee2aebe341e91c61e"} Sep 09 20:55:02.482735 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:02.482297 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" event={"ID":"14a43885-6f91-41fc-bafe-08fec415b79e","Type":"ContainerStarted","Data":"99f067f3c9b5bbfe1da5b30920b4a622af24725e1570ac657d1a17447a40de37"} Sep 09 20:55:02.497082 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:02.497029 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-78888988f7-jf6jb" podStartSLOduration=19.497010402 podStartE2EDuration="19.497010402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:00.498843295 +0000 UTC m=+277.338042432" watchObservedRunningTime="2026-09-09 20:55:02.497010402 +0000 UTC m=+279.336209544" Sep 09 20:55:02.497490 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:02.497466 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-dvhjx" podStartSLOduration=17.793817728 podStartE2EDuration="19.497457404s" totalImagesPullingTime="1.703639676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:02.496704596 +0000 UTC m=+279.335903740" watchObservedRunningTime="2026-09-09 20:55:02.497457404 +0000 UTC m=+279.336656534" Sep 09 20:55:04.158327 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:55:04.158284 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8wlxq" podUID="df144005-7adc-4de3-8c5b-37584f6ba534" Sep 09 20:55:04.450282 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:04.450187 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:55:04.450282 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:04.450259 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:55:04.450698 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:04.450679 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42671ab6-cd6a-4a2d-8494-828a140848f6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:55:04.452580 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:04.452562 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42671ab6-cd6a-4a2d-8494-828a140848f6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dct65\" (UID: \"42671ab6-cd6a-4a2d-8494-828a140848f6\") " pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:55:04.488619 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:04.488590 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8wlxq" Sep 09 20:55:04.527992 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:04.527967 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dct65" Sep 09 20:55:04.804575 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:04.804545 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dct65"] Sep 09 20:55:04.808071 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:55:04.808047 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42671ab6_cd6a_4a2d_8494_828a140848f6.slice/crio-1dbcdc04b4761e7ce371414aeb6938c51284446a82cbaf77ed92ebff34119bac WatchSource:0}: Error finding container 1dbcdc04b4761e7ce371414aeb6938c51284446a82cbaf77ed92ebff34119bac: Status 404 returned error can't find the container with id 1dbcdc04b4761e7ce371414aeb6938c51284446a82cbaf77ed92ebff34119bac Sep 09 20:55:05.492913 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:05.492884 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dct65" event={"ID":"42671ab6-cd6a-4a2d-8494-828a140848f6","Type":"ContainerStarted","Data":"42a07c6d0cf626388090742cd64d350c62f84fd9097ae6b31ae6747e1e68eda7"} Sep 09 20:55:05.493252 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:05.492950 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dct65" event={"ID":"42671ab6-cd6a-4a2d-8494-828a140848f6","Type":"ContainerStarted","Data":"1dbcdc04b4761e7ce371414aeb6938c51284446a82cbaf77ed92ebff34119bac"} Sep 09 20:55:06.504516 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:06.504481 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dct65" event={"ID":"42671ab6-cd6a-4a2d-8494-828a140848f6","Type":"ContainerStarted","Data":"f019f70166b40db271445b66f2e32c6d135eee41bedd5e7764fd99cac0597618"} Sep 09 20:55:07.374700 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:07.374668 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:55:07.377124 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:07.377104 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df144005-7adc-4de3-8c5b-37584f6ba534-metrics-tls\") pod \"dns-default-8wlxq\" (UID: \"df144005-7adc-4de3-8c5b-37584f6ba534\") " pod="openshift-dns/dns-default-8wlxq" Sep 09 20:55:07.492447 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:07.492400 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wtvxz\"" Sep 09 20:55:07.500073 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:07.500057 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8wlxq" Sep 09 20:55:07.510088 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:07.510069 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dct65" event={"ID":"42671ab6-cd6a-4a2d-8494-828a140848f6","Type":"ContainerStarted","Data":"3fd952b214650972f78ceca4eeb461e721d189c2e5242ee83cc68aad31843dfa"} Sep 09 20:55:07.530050 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:07.529996 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dct65" podStartSLOduration=17.575921188 podStartE2EDuration="19.529980713s" totalImagesPullingTime="1.954059525s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:54:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:07.528567901 +0000 UTC m=+284.367767051" watchObservedRunningTime="2026-09-09 20:55:07.529980713 +0000 UTC m=+284.369179842" Sep 09 20:55:07.775028 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:07.774998 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8wlxq"] Sep 09 20:55:07.778058 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:55:07.778028 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf144005_7adc_4de3_8c5b_37584f6ba534.slice/crio-1e8d850af136f66460358456c4d822df494ef3c294b2e25c0e7f3edda075433c WatchSource:0}: Error finding container 1e8d850af136f66460358456c4d822df494ef3c294b2e25c0e7f3edda075433c: Status 404 returned error can't find the container with id 1e8d850af136f66460358456c4d822df494ef3c294b2e25c0e7f3edda075433c Sep 09 20:55:08.515569 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:08.515478 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8wlxq" event={"ID":"df144005-7adc-4de3-8c5b-37584f6ba534","Type":"ContainerStarted","Data":"1e8d850af136f66460358456c4d822df494ef3c294b2e25c0e7f3edda075433c"} Sep 09 20:55:09.521012 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:09.520984 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8wlxq" event={"ID":"df144005-7adc-4de3-8c5b-37584f6ba534","Type":"ContainerStarted","Data":"9aae6938bc255e7b9f5b1ed7723d39d58bd25b07cbfb0b66a0607bee8b9fd807"} Sep 09 20:55:09.521420 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:09.521030 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8wlxq" event={"ID":"df144005-7adc-4de3-8c5b-37584f6ba534","Type":"ContainerStarted","Data":"f97a75972f444fa89201bfd6a66452f05e07b780a289f2ad46062d09d9844b04"} Sep 09 20:55:09.521420 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:09.521085 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8wlxq" Sep 09 20:55:09.537476 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:09.537424 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8wlxq" podStartSLOduration=250.992588052 podStartE2EDuration="4m12.537408697s" totalImagesPullingTime="1.544820645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:50:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:09.537101537 +0000 UTC m=+286.376300667" watchObservedRunningTime="2026-09-09 20:55:09.537408697 +0000 UTC m=+286.376607808" Sep 09 20:55:14.582856 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.582820 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-2689n"] Sep 09 20:55:14.585687 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.585666 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-2689n" Sep 09 20:55:14.589280 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.589260 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 09 20:55:14.589435 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.589418 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 09 20:55:14.589577 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.589557 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-n2zmm\"" Sep 09 20:55:14.593257 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.593213 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78888988f7-jf6jb"] Sep 09 20:55:14.597454 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.597430 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-2689n"] Sep 09 20:55:14.708415 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.708372 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d956956fc-bjp2t"] Sep 09 20:55:14.711031 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.711015 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.726753 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.726727 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8p2k\" (UniqueName: \"kubernetes.io/projected/e37d004a-3684-4ca0-ba67-c0da1a62496a-kube-api-access-c8p2k\") pod \"downloads-85d98c998f-2689n\" (UID: \"e37d004a-3684-4ca0-ba67-c0da1a62496a\") " pod="openshift-console/downloads-85d98c998f-2689n" Sep 09 20:55:14.737378 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.737351 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d956956fc-bjp2t"] Sep 09 20:55:14.827760 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.827723 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f9f23ca0-784b-4f62-abea-cf8ac9060dad-registry-certificates\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.827760 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.827765 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f9f23ca0-784b-4f62-abea-cf8ac9060dad-registry-tls\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.827984 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.827790 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f9f23ca0-784b-4f62-abea-cf8ac9060dad-image-registry-private-configuration\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.827984 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.827865 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f9f23ca0-784b-4f62-abea-cf8ac9060dad-trusted-ca\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.827984 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.827902 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9f23ca0-784b-4f62-abea-cf8ac9060dad-bound-sa-token\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.827984 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.827924 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d8vv\" (UniqueName: \"kubernetes.io/projected/f9f23ca0-784b-4f62-abea-cf8ac9060dad-kube-api-access-5d8vv\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.827984 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.827947 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8p2k\" (UniqueName: \"kubernetes.io/projected/e37d004a-3684-4ca0-ba67-c0da1a62496a-kube-api-access-c8p2k\") pod \"downloads-85d98c998f-2689n\" (UID: \"e37d004a-3684-4ca0-ba67-c0da1a62496a\") " pod="openshift-console/downloads-85d98c998f-2689n" Sep 09 20:55:14.827984 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.827966 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f9f23ca0-784b-4f62-abea-cf8ac9060dad-ca-trust-extracted\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.828193 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.828029 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f9f23ca0-784b-4f62-abea-cf8ac9060dad-installation-pull-secrets\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.837048 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.836979 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8p2k\" (UniqueName: \"kubernetes.io/projected/e37d004a-3684-4ca0-ba67-c0da1a62496a-kube-api-access-c8p2k\") pod \"downloads-85d98c998f-2689n\" (UID: \"e37d004a-3684-4ca0-ba67-c0da1a62496a\") " pod="openshift-console/downloads-85d98c998f-2689n" Sep 09 20:55:14.895338 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.895296 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-2689n" Sep 09 20:55:14.928806 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.928768 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f9f23ca0-784b-4f62-abea-cf8ac9060dad-trusted-ca\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.928806 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.928809 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9f23ca0-784b-4f62-abea-cf8ac9060dad-bound-sa-token\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.929048 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.928836 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d8vv\" (UniqueName: \"kubernetes.io/projected/f9f23ca0-784b-4f62-abea-cf8ac9060dad-kube-api-access-5d8vv\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.929048 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.928863 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f9f23ca0-784b-4f62-abea-cf8ac9060dad-ca-trust-extracted\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.929048 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.928918 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f9f23ca0-784b-4f62-abea-cf8ac9060dad-installation-pull-secrets\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.929048 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.928963 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f9f23ca0-784b-4f62-abea-cf8ac9060dad-registry-certificates\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.929048 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.929002 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f9f23ca0-784b-4f62-abea-cf8ac9060dad-registry-tls\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.929048 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.929034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f9f23ca0-784b-4f62-abea-cf8ac9060dad-image-registry-private-configuration\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.929429 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.929403 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f9f23ca0-784b-4f62-abea-cf8ac9060dad-ca-trust-extracted\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.929886 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.929865 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f9f23ca0-784b-4f62-abea-cf8ac9060dad-trusted-ca\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.930158 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.929905 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f9f23ca0-784b-4f62-abea-cf8ac9060dad-registry-certificates\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.932206 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.932149 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f9f23ca0-784b-4f62-abea-cf8ac9060dad-registry-tls\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.932330 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.932260 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f9f23ca0-784b-4f62-abea-cf8ac9060dad-image-registry-private-configuration\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.932330 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.932289 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f9f23ca0-784b-4f62-abea-cf8ac9060dad-installation-pull-secrets\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.937640 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.937621 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9f23ca0-784b-4f62-abea-cf8ac9060dad-bound-sa-token\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:14.938060 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:14.938009 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d8vv\" (UniqueName: \"kubernetes.io/projected/f9f23ca0-784b-4f62-abea-cf8ac9060dad-kube-api-access-5d8vv\") pod \"image-registry-7d956956fc-bjp2t\" (UID: \"f9f23ca0-784b-4f62-abea-cf8ac9060dad\") " pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:15.019203 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:15.019168 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:15.166496 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:15.166424 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-2689n"] Sep 09 20:55:15.169655 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:55:15.169633 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode37d004a_3684_4ca0_ba67_c0da1a62496a.slice/crio-d4b6b4a504a49c3d02fefc17deeefa891d20a23c502ab6a13bb4e8cc2f3c5368 WatchSource:0}: Error finding container d4b6b4a504a49c3d02fefc17deeefa891d20a23c502ab6a13bb4e8cc2f3c5368: Status 404 returned error can't find the container with id d4b6b4a504a49c3d02fefc17deeefa891d20a23c502ab6a13bb4e8cc2f3c5368 Sep 09 20:55:15.292717 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:15.292684 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d956956fc-bjp2t"] Sep 09 20:55:15.296260 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:55:15.296216 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f23ca0_784b_4f62_abea_cf8ac9060dad.slice/crio-04680e068140473238da5d1ca26a64b993a017275d62538c77224aa0caf98ac9 WatchSource:0}: Error finding container 04680e068140473238da5d1ca26a64b993a017275d62538c77224aa0caf98ac9: Status 404 returned error can't find the container with id 04680e068140473238da5d1ca26a64b993a017275d62538c77224aa0caf98ac9 Sep 09 20:55:15.541305 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:15.541268 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-2689n" event={"ID":"e37d004a-3684-4ca0-ba67-c0da1a62496a","Type":"ContainerStarted","Data":"d4b6b4a504a49c3d02fefc17deeefa891d20a23c502ab6a13bb4e8cc2f3c5368"} Sep 09 20:55:15.542855 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:15.542831 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" event={"ID":"f9f23ca0-784b-4f62-abea-cf8ac9060dad","Type":"ContainerStarted","Data":"a58412ccf047af0eaeecf27115fc7c41595c1dbc2fd8d61fb0cadb170d517e78"} Sep 09 20:55:15.542982 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:15.542873 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" event={"ID":"f9f23ca0-784b-4f62-abea-cf8ac9060dad","Type":"ContainerStarted","Data":"04680e068140473238da5d1ca26a64b993a017275d62538c77224aa0caf98ac9"} Sep 09 20:55:15.542982 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:15.542951 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:15.562806 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:15.562758 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" podStartSLOduration=1.562744682 podStartE2EDuration="1.562744682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:55:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:15.562148981 +0000 UTC m=+292.401348110" watchObservedRunningTime="2026-09-09 20:55:15.562744682 +0000 UTC m=+292.401943811" Sep 09 20:55:17.634071 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:55:17.634020 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-lqd29" podUID="ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5" Sep 09 20:55:19.527192 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:19.527161 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8wlxq" Sep 09 20:55:23.516424 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:23.516395 2562 kubelet.go:1648] "Image garbage collection succeeded" Sep 09 20:55:24.599297 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:24.599271 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:55:25.906143 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.906103 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz"] Sep 09 20:55:25.909448 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.909427 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:25.912461 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.912441 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 09 20:55:25.912584 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.912467 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 09 20:55:25.912584 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.912478 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 09 20:55:25.912704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.912628 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 09 20:55:25.913085 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.913065 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 09 20:55:25.913172 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.913089 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nwwjk\"" Sep 09 20:55:25.913253 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.913183 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 09 20:55:25.923414 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.923387 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7e332e8-9425-4403-8a37-60af9a8058ec-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:25.923517 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.923501 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d7e332e8-9425-4403-8a37-60af9a8058ec-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:25.923605 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.923569 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7e332e8-9425-4403-8a37-60af9a8058ec-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:25.923605 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.923652 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d7e332e8-9425-4403-8a37-60af9a8058ec-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:25.923605 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.923688 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d7e332e8-9425-4403-8a37-60af9a8058ec-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:25.923605 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.923711 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bp9b\" (UniqueName: \"kubernetes.io/projected/d7e332e8-9425-4403-8a37-60af9a8058ec-kube-api-access-4bp9b\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:25.926001 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.925978 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz"] Sep 09 20:55:25.930921 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.930897 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-24fjb"] Sep 09 20:55:25.934276 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.934257 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:25.936466 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.936445 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 09 20:55:25.936575 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.936546 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 09 20:55:25.936918 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.936899 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 09 20:55:25.937019 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:25.936983 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-95jk5\"" Sep 09 20:55:26.024524 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.024482 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0fca05bf-c7fd-48ca-b5da-53be00d48a19-node-exporter-accelerators-collector-config\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.024712 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.024526 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7e332e8-9425-4403-8a37-60af9a8058ec-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:26.024712 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.024603 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d7e332e8-9425-4403-8a37-60af9a8058ec-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:26.024825 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.024711 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7e332e8-9425-4403-8a37-60af9a8058ec-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:26.024825 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.024745 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0fca05bf-c7fd-48ca-b5da-53be00d48a19-node-exporter-wtmp\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.024825 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.024763 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0fca05bf-c7fd-48ca-b5da-53be00d48a19-sys\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.024825 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.024787 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfbzk\" (UniqueName: \"kubernetes.io/projected/0fca05bf-c7fd-48ca-b5da-53be00d48a19-kube-api-access-hfbzk\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.025036 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.024854 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0fca05bf-c7fd-48ca-b5da-53be00d48a19-node-exporter-tls\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.025036 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:55:26.024886 2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Sep 09 20:55:26.025036 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.024917 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d7e332e8-9425-4403-8a37-60af9a8058ec-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:26.025036 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:55:26.024947 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7e332e8-9425-4403-8a37-60af9a8058ec-kube-state-metrics-tls podName:d7e332e8-9425-4403-8a37-60af9a8058ec nodeName:}" failed. No retries permitted until 2026-09-09 20:55:26.524927957 +0000 UTC m=+303.364127073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/d7e332e8-9425-4403-8a37-60af9a8058ec-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-bsnjz" (UID: "d7e332e8-9425-4403-8a37-60af9a8058ec") : secret "kube-state-metrics-tls" not found Sep 09 20:55:26.025036 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.024995 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0fca05bf-c7fd-48ca-b5da-53be00d48a19-root\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.025036 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.025023 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d7e332e8-9425-4403-8a37-60af9a8058ec-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:26.025335 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.025029 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d7e332e8-9425-4403-8a37-60af9a8058ec-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:26.025335 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.025072 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bp9b\" (UniqueName: \"kubernetes.io/projected/d7e332e8-9425-4403-8a37-60af9a8058ec-kube-api-access-4bp9b\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:26.025335 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.025097 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fca05bf-c7fd-48ca-b5da-53be00d48a19-metrics-client-ca\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.025335 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.025126 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0fca05bf-c7fd-48ca-b5da-53be00d48a19-node-exporter-textfile\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.025335 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.025155 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fca05bf-c7fd-48ca-b5da-53be00d48a19-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.025599 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.025446 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7e332e8-9425-4403-8a37-60af9a8058ec-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:26.025689 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.025669 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d7e332e8-9425-4403-8a37-60af9a8058ec-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:26.027677 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.027657 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d7e332e8-9425-4403-8a37-60af9a8058ec-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:26.035284 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.035230 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bp9b\" (UniqueName: \"kubernetes.io/projected/d7e332e8-9425-4403-8a37-60af9a8058ec-kube-api-access-4bp9b\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:26.125605 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.125572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0fca05bf-c7fd-48ca-b5da-53be00d48a19-node-exporter-wtmp\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.125781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.125616 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0fca05bf-c7fd-48ca-b5da-53be00d48a19-sys\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.125781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.125640 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfbzk\" (UniqueName: \"kubernetes.io/projected/0fca05bf-c7fd-48ca-b5da-53be00d48a19-kube-api-access-hfbzk\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.125781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.125671 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0fca05bf-c7fd-48ca-b5da-53be00d48a19-node-exporter-tls\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.125781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.125730 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0fca05bf-c7fd-48ca-b5da-53be00d48a19-root\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.125781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.125752 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0fca05bf-c7fd-48ca-b5da-53be00d48a19-node-exporter-wtmp\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.125781 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.125756 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fca05bf-c7fd-48ca-b5da-53be00d48a19-metrics-client-ca\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.126368 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.125796 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0fca05bf-c7fd-48ca-b5da-53be00d48a19-sys\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.126368 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.125807 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0fca05bf-c7fd-48ca-b5da-53be00d48a19-node-exporter-textfile\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.126368 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.125872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fca05bf-c7fd-48ca-b5da-53be00d48a19-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.126368 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.125904 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0fca05bf-c7fd-48ca-b5da-53be00d48a19-node-exporter-accelerators-collector-config\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.126368 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.126101 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0fca05bf-c7fd-48ca-b5da-53be00d48a19-node-exporter-textfile\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.126604 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.126436 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fca05bf-c7fd-48ca-b5da-53be00d48a19-metrics-client-ca\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.126604 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.126483 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0fca05bf-c7fd-48ca-b5da-53be00d48a19-root\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.126604 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.126514 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0fca05bf-c7fd-48ca-b5da-53be00d48a19-node-exporter-accelerators-collector-config\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.129530 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.129482 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fca05bf-c7fd-48ca-b5da-53be00d48a19-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.129684 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.129663 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0fca05bf-c7fd-48ca-b5da-53be00d48a19-node-exporter-tls\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.135280 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.135260 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfbzk\" (UniqueName: \"kubernetes.io/projected/0fca05bf-c7fd-48ca-b5da-53be00d48a19-kube-api-access-hfbzk\") pod \"node-exporter-24fjb\" (UID: \"0fca05bf-c7fd-48ca-b5da-53be00d48a19\") " pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.245509 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.245477 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-24fjb" Sep 09 20:55:26.528065 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.527985 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7e332e8-9425-4403-8a37-60af9a8058ec-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:26.530714 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.530687 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7e332e8-9425-4403-8a37-60af9a8058ec-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-bsnjz\" (UID: \"d7e332e8-9425-4403-8a37-60af9a8058ec\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:26.821144 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.821062 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" Sep 09 20:55:26.977786 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.977749 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 20:55:26.982118 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.982097 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:26.986632 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.986610 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 09 20:55:26.987298 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.986951 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 09 20:55:26.987298 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.986972 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 09 20:55:26.987298 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.986982 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-w6jhd\"" Sep 09 20:55:26.987298 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.987009 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 09 20:55:26.987298 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.986977 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 09 20:55:26.987298 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.987103 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 09 20:55:26.987717 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.987498 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 09 20:55:26.987717 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.987519 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 09 20:55:26.987811 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.987793 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 09 20:55:26.997345 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:26.997324 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 20:55:27.033420 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.033381 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-config-out\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.033597 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.033444 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7gxq\" (UniqueName: \"kubernetes.io/projected/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-kube-api-access-d7gxq\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.033597 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.033512 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.033597 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.033552 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.033784 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.033606 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.033784 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.033635 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-web-config\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.033784 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.033690 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-config-volume\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.033784 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.033717 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.033784 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.033752 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.034010 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.033783 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.034010 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.033812 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.034010 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.033843 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.034010 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.033906 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.134928 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.134830 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.134928 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.134875 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.134928 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.134890 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-web-config\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.134928 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.134915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-config-volume\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.134928 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.134930 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.135338 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.134958 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.135338 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.134977 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.135338 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:55:27.134999 2562 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 09 20:55:27.135338 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:55:27.135073 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-main-tls podName:efc9a8cb-7dba-4ddb-9435-0d4d07d0305f nodeName:}" failed. No retries permitted until 2026-09-09 20:55:27.63505233 +0000 UTC m=+304.474251453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f") : secret "alertmanager-main-tls" not found Sep 09 20:55:27.135338 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:55:27.135102 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-alertmanager-trusted-ca-bundle podName:efc9a8cb-7dba-4ddb-9435-0d4d07d0305f nodeName:}" failed. No retries permitted until 2026-09-09 20:55:27.635083285 +0000 UTC m=+304.474282397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f") : configmap references non-existent config key: ca-bundle.crt Sep 09 20:55:27.135338 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.135003 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.135649 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.135341 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.135649 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.135377 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.135649 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.135435 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.135649 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.135441 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-config-out\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.135649 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.135506 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7gxq\" (UniqueName: \"kubernetes.io/projected/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-kube-api-access-d7gxq\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.135649 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.135540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.136581 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.136226 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.138441 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.138416 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.138749 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.138714 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-config-volume\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.138864 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.138793 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-web-config\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.138985 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.138940 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.139588 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.139565 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.139789 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.139770 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-config-out\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.139968 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.139927 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.140302 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.140283 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.146973 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.146952 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7gxq\" (UniqueName: \"kubernetes.io/projected/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-kube-api-access-d7gxq\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.639402 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.639368 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.639589 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.639476 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.640348 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.640225 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.642256 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.642212 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:27.894064 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:27.893984 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:55:30.633121 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:30.633084 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:55:30.843825 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:55:30.843794 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fca05bf_c7fd_48ca_b5da_53be00d48a19.slice/crio-676575b8cdc37ca4ddecce44498fb7cca6bba24a38d6292b142b0db423de66af WatchSource:0}: Error finding container 676575b8cdc37ca4ddecce44498fb7cca6bba24a38d6292b142b0db423de66af: Status 404 returned error can't find the container with id 676575b8cdc37ca4ddecce44498fb7cca6bba24a38d6292b142b0db423de66af Sep 09 20:55:30.846567 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:30.846539 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 20:55:31.119028 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:31.118993 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz"] Sep 09 20:55:31.122093 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:55:31.122049 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e332e8_9425_4403_8a37_60af9a8058ec.slice/crio-3182881a4d6b3ee1e4eaf70cd8147eca9a7a92aa4887f512baca5d6b74b7e90f WatchSource:0}: Error finding container 3182881a4d6b3ee1e4eaf70cd8147eca9a7a92aa4887f512baca5d6b74b7e90f: Status 404 returned error can't find the container with id 3182881a4d6b3ee1e4eaf70cd8147eca9a7a92aa4887f512baca5d6b74b7e90f Sep 09 20:55:31.125769 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:31.125744 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 20:55:31.130490 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:55:31.130466 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefc9a8cb_7dba_4ddb_9435_0d4d07d0305f.slice/crio-66300587d2e9cb6cbca362b10dc150720ba826829b8b5d0d6af276c4d443d50f WatchSource:0}: Error finding container 66300587d2e9cb6cbca362b10dc150720ba826829b8b5d0d6af276c4d443d50f: Status 404 returned error can't find the container with id 66300587d2e9cb6cbca362b10dc150720ba826829b8b5d0d6af276c4d443d50f Sep 09 20:55:31.609380 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:31.609336 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-2689n" event={"ID":"e37d004a-3684-4ca0-ba67-c0da1a62496a","Type":"ContainerStarted","Data":"82862abd8627b0940566da3ba50dda3a2973d1141d78a4c2b25b7110e65fb478"} Sep 09 20:55:31.612981 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:31.612936 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-2689n" Sep 09 20:55:31.614423 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:31.614383 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" event={"ID":"d7e332e8-9425-4403-8a37-60af9a8058ec","Type":"ContainerStarted","Data":"3182881a4d6b3ee1e4eaf70cd8147eca9a7a92aa4887f512baca5d6b74b7e90f"} Sep 09 20:55:31.616582 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:31.616547 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24fjb" event={"ID":"0fca05bf-c7fd-48ca-b5da-53be00d48a19","Type":"ContainerStarted","Data":"676575b8cdc37ca4ddecce44498fb7cca6bba24a38d6292b142b0db423de66af"} Sep 09 20:55:31.619180 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:31.619093 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f","Type":"ContainerStarted","Data":"66300587d2e9cb6cbca362b10dc150720ba826829b8b5d0d6af276c4d443d50f"} Sep 09 20:55:31.623660 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:31.623615 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-2689n" Sep 09 20:55:31.667422 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:31.667126 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-2689n" podStartSLOduration=1.901633095 podStartE2EDuration="17.66710035s" totalImagesPullingTime="15.765467255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:55:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:31.641976899 +0000 UTC m=+308.481176030" watchObservedRunningTime="2026-09-09 20:55:31.66710035 +0000 UTC m=+308.506299482" Sep 09 20:55:33.630701 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:33.630663 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" event={"ID":"d7e332e8-9425-4403-8a37-60af9a8058ec","Type":"ContainerStarted","Data":"ca88ea1f0807e803b93207b4c5870d27ec969de2fb3e6458dace53010a832d29"} Sep 09 20:55:33.641259 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:33.640928 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24fjb" event={"ID":"0fca05bf-c7fd-48ca-b5da-53be00d48a19","Type":"ContainerStarted","Data":"b017ea1b07d12fb0f247c3d6e220f2daaf6b82652435c2b97f12ad9634429ea7"} Sep 09 20:55:34.648991 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:34.648752 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" event={"ID":"d7e332e8-9425-4403-8a37-60af9a8058ec","Type":"ContainerStarted","Data":"352b1ce18ae97ecb2e7b19fe7f2189d3d6b4942428809244fb926721de05460a"} Sep 09 20:55:34.648991 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:34.648827 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" event={"ID":"d7e332e8-9425-4403-8a37-60af9a8058ec","Type":"ContainerStarted","Data":"9fbf836fef90ba9d7be2c250386f2f57b7a06e08e63475d80fedb375e06d0747"} Sep 09 20:55:34.650898 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:34.650871 2562 generic.go:356] "Generic (PLEG): container finished" podID="0fca05bf-c7fd-48ca-b5da-53be00d48a19" containerID="b017ea1b07d12fb0f247c3d6e220f2daaf6b82652435c2b97f12ad9634429ea7" exitCode=0 Sep 09 20:55:34.651031 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:34.650955 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24fjb" event={"ID":"0fca05bf-c7fd-48ca-b5da-53be00d48a19","Type":"ContainerDied","Data":"b017ea1b07d12fb0f247c3d6e220f2daaf6b82652435c2b97f12ad9634429ea7"} Sep 09 20:55:34.652937 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:34.652908 2562 generic.go:356] "Generic (PLEG): container finished" podID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerID="9e227a643920b60d7987d48ebafbcf3ce17d0a1fc168502e8f4eadbc61a257da" exitCode=0 Sep 09 20:55:34.653067 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:34.652990 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f","Type":"ContainerDied","Data":"9e227a643920b60d7987d48ebafbcf3ce17d0a1fc168502e8f4eadbc61a257da"} Sep 09 20:55:34.672065 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:34.672020 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-bsnjz" podStartSLOduration=7.3326458930000005 podStartE2EDuration="9.67200477s" totalImagesPullingTime="2.339358877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:34.669883232 +0000 UTC m=+311.509082390" watchObservedRunningTime="2026-09-09 20:55:34.67200477 +0000 UTC m=+311.511203899" Sep 09 20:55:35.660529 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:35.660475 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24fjb" event={"ID":"0fca05bf-c7fd-48ca-b5da-53be00d48a19","Type":"ContainerStarted","Data":"3963f54f81eb3ffbbf1bfa576b6f069bd8fe3592835034c59d5bb7b2fa2f7bc9"} Sep 09 20:55:35.661053 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:35.660599 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24fjb" event={"ID":"0fca05bf-c7fd-48ca-b5da-53be00d48a19","Type":"ContainerStarted","Data":"cd98fdf058eb3241d58136f33dcf33381e855705eab06a79d6de7365da0f27ac"} Sep 09 20:55:35.682252 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:35.682189 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-24fjb" podStartSLOduration=7.936482945 podStartE2EDuration="10.682167864s" totalImagesPullingTime="2.614773919s" totalInitContainerRuntime="130.911ms" isStatefulPod=true podCreationTimestamp="2026-09-09 20:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:35.680381304 +0000 UTC m=+312.519580455" watchObservedRunningTime="2026-09-09 20:55:35.682167864 +0000 UTC m=+312.521366995" Sep 09 20:55:36.551180 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:36.551151 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d956956fc-bjp2t" Sep 09 20:55:37.674788 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:37.674698 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f","Type":"ContainerStarted","Data":"fda4e913449d0429ae06b18978eb9aeca8b90ab864f3cf757fecb9ebc492e1cb"} Sep 09 20:55:37.674788 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:37.674785 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f","Type":"ContainerStarted","Data":"7b027767d15768cd506179f7061873acb63959abc6d37ad97dd6e4d09057604e"} Sep 09 20:55:37.675220 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:37.674832 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f","Type":"ContainerStarted","Data":"bfed4077fab976a709f08932a89a7a830c93d16c277de0754b1dd6194df3e9da"} Sep 09 20:55:37.675220 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:37.674859 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f","Type":"ContainerStarted","Data":"d82126b7ee58e7dfddc12cbdc61567eb0e64c2050e53b7cf49b225ba0283290a"} Sep 09 20:55:37.675220 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:37.674882 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f","Type":"ContainerStarted","Data":"5bf7e6bed2a55a89932bff870543d6697c9614df4e6b8a5486b82c1420d2bbaa"} Sep 09 20:55:39.613914 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:39.613862 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-78888988f7-jf6jb" podUID="372b1efb-f45d-4dae-9d3a-2070ad8099b2" containerName="registry" containerID="cri-o://559f265396d09fc9bee7e0276938f63fb72ef8ec24fb6b5f1b10187f63b23559" gracePeriod=30 Sep 09 20:55:40.022025 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.021998 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:55:40.061199 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.061163 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls\") pod \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " Sep 09 20:55:40.061399 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.061223 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-certificates\") pod \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " Sep 09 20:55:40.061399 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.061284 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-bound-sa-token\") pod \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " Sep 09 20:55:40.061399 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.061310 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/372b1efb-f45d-4dae-9d3a-2070ad8099b2-ca-trust-extracted\") pod \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " Sep 09 20:55:40.061610 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.061405 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/372b1efb-f45d-4dae-9d3a-2070ad8099b2-installation-pull-secrets\") pod \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " Sep 09 20:55:40.061610 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.061431 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/372b1efb-f45d-4dae-9d3a-2070ad8099b2-trusted-ca\") pod \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " Sep 09 20:55:40.061610 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.061455 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzwgh\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-kube-api-access-dzwgh\") pod \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " Sep 09 20:55:40.061610 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.061494 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/372b1efb-f45d-4dae-9d3a-2070ad8099b2-image-registry-private-configuration\") pod \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\" (UID: \"372b1efb-f45d-4dae-9d3a-2070ad8099b2\") " Sep 09 20:55:40.062350 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.062317 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "372b1efb-f45d-4dae-9d3a-2070ad8099b2" (UID: "372b1efb-f45d-4dae-9d3a-2070ad8099b2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:55:40.062684 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.062660 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/372b1efb-f45d-4dae-9d3a-2070ad8099b2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "372b1efb-f45d-4dae-9d3a-2070ad8099b2" (UID: "372b1efb-f45d-4dae-9d3a-2070ad8099b2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:55:40.064893 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.064830 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/372b1efb-f45d-4dae-9d3a-2070ad8099b2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "372b1efb-f45d-4dae-9d3a-2070ad8099b2" (UID: "372b1efb-f45d-4dae-9d3a-2070ad8099b2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:55:40.064893 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.064854 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "372b1efb-f45d-4dae-9d3a-2070ad8099b2" (UID: "372b1efb-f45d-4dae-9d3a-2070ad8099b2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:55:40.065264 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.065216 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "372b1efb-f45d-4dae-9d3a-2070ad8099b2" (UID: "372b1efb-f45d-4dae-9d3a-2070ad8099b2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:55:40.069601 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.069572 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/372b1efb-f45d-4dae-9d3a-2070ad8099b2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "372b1efb-f45d-4dae-9d3a-2070ad8099b2" (UID: "372b1efb-f45d-4dae-9d3a-2070ad8099b2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:55:40.072184 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.072158 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-kube-api-access-dzwgh" (OuterVolumeSpecName: "kube-api-access-dzwgh") pod "372b1efb-f45d-4dae-9d3a-2070ad8099b2" (UID: "372b1efb-f45d-4dae-9d3a-2070ad8099b2"). InnerVolumeSpecName "kube-api-access-dzwgh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:55:40.073390 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.073367 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/372b1efb-f45d-4dae-9d3a-2070ad8099b2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "372b1efb-f45d-4dae-9d3a-2070ad8099b2" (UID: "372b1efb-f45d-4dae-9d3a-2070ad8099b2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 20:55:40.162690 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.162607 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-bound-sa-token\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:55:40.162690 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.162637 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/372b1efb-f45d-4dae-9d3a-2070ad8099b2-ca-trust-extracted\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:55:40.162690 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.162647 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/372b1efb-f45d-4dae-9d3a-2070ad8099b2-installation-pull-secrets\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:55:40.162690 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.162656 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/372b1efb-f45d-4dae-9d3a-2070ad8099b2-trusted-ca\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:55:40.162690 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.162664 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzwgh\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-kube-api-access-dzwgh\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:55:40.162690 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.162673 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/372b1efb-f45d-4dae-9d3a-2070ad8099b2-image-registry-private-configuration\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:55:40.162690 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.162682 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-tls\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:55:40.162690 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.162690 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/372b1efb-f45d-4dae-9d3a-2070ad8099b2-registry-certificates\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:55:40.688304 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.688268 2562 generic.go:356] "Generic (PLEG): container finished" podID="372b1efb-f45d-4dae-9d3a-2070ad8099b2" containerID="559f265396d09fc9bee7e0276938f63fb72ef8ec24fb6b5f1b10187f63b23559" exitCode=0 Sep 09 20:55:40.688804 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.688332 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78888988f7-jf6jb" Sep 09 20:55:40.688804 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.688359 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78888988f7-jf6jb" event={"ID":"372b1efb-f45d-4dae-9d3a-2070ad8099b2","Type":"ContainerDied","Data":"559f265396d09fc9bee7e0276938f63fb72ef8ec24fb6b5f1b10187f63b23559"} Sep 09 20:55:40.688804 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.688431 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78888988f7-jf6jb" event={"ID":"372b1efb-f45d-4dae-9d3a-2070ad8099b2","Type":"ContainerDied","Data":"a0562858f386f7562037d870d4c68439d8b68767222f7547a3e826876c706d3b"} Sep 09 20:55:40.688804 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.688462 2562 scope.go:117] "RemoveContainer" containerID="559f265396d09fc9bee7e0276938f63fb72ef8ec24fb6b5f1b10187f63b23559" Sep 09 20:55:40.692198 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.692166 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f","Type":"ContainerStarted","Data":"33a5fc73e01b03c8f949a2d5b971d2cdaad5e60198a4f85415e1c95d4d9d4b72"} Sep 09 20:55:40.698275 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.698252 2562 scope.go:117] "RemoveContainer" containerID="559f265396d09fc9bee7e0276938f63fb72ef8ec24fb6b5f1b10187f63b23559" Sep 09 20:55:40.698572 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:55:40.698553 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"559f265396d09fc9bee7e0276938f63fb72ef8ec24fb6b5f1b10187f63b23559\": container with ID starting with 559f265396d09fc9bee7e0276938f63fb72ef8ec24fb6b5f1b10187f63b23559 not found: ID does not exist" containerID="559f265396d09fc9bee7e0276938f63fb72ef8ec24fb6b5f1b10187f63b23559" Sep 09 20:55:40.698637 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.698583 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"559f265396d09fc9bee7e0276938f63fb72ef8ec24fb6b5f1b10187f63b23559"} err="failed to get container status \"559f265396d09fc9bee7e0276938f63fb72ef8ec24fb6b5f1b10187f63b23559\": rpc error: code = NotFound desc = could not find container \"559f265396d09fc9bee7e0276938f63fb72ef8ec24fb6b5f1b10187f63b23559\": container with ID starting with 559f265396d09fc9bee7e0276938f63fb72ef8ec24fb6b5f1b10187f63b23559 not found: ID does not exist" Sep 09 20:55:40.722723 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.722676 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=7.880619867 podStartE2EDuration="14.722658496s" totalImagesPullingTime="6.810593529s" totalInitContainerRuntime="31.4451ms" isStatefulPod=false podCreationTimestamp="2026-09-09 20:55:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:40.720530032 +0000 UTC m=+317.559729162" watchObservedRunningTime="2026-09-09 20:55:40.722658496 +0000 UTC m=+317.561857625" Sep 09 20:55:40.734515 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.734487 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78888988f7-jf6jb"] Sep 09 20:55:40.737255 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:40.737212 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-78888988f7-jf6jb"] Sep 09 20:55:41.638136 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:41.638098 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="372b1efb-f45d-4dae-9d3a-2070ad8099b2" path="/var/lib/kubelet/pods/372b1efb-f45d-4dae-9d3a-2070ad8099b2/volumes" Sep 09 20:55:52.736903 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:52.736867 2562 generic.go:356] "Generic (PLEG): container finished" podID="c9eaaaa8-54b7-4c98-b4a6-643a2826f899" containerID="2537a53f3c1c3317ba774ddd938cb817400e1073038af95925c86910ecb119cf" exitCode=0 Sep 09 20:55:52.737338 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:52.736941 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s" event={"ID":"c9eaaaa8-54b7-4c98-b4a6-643a2826f899","Type":"ContainerDied","Data":"2537a53f3c1c3317ba774ddd938cb817400e1073038af95925c86910ecb119cf"} Sep 09 20:55:52.737338 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:52.737229 2562 scope.go:117] "RemoveContainer" containerID="2537a53f3c1c3317ba774ddd938cb817400e1073038af95925c86910ecb119cf" Sep 09 20:55:53.741849 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:55:53.741815 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-r829s" event={"ID":"c9eaaaa8-54b7-4c98-b4a6-643a2826f899","Type":"ContainerStarted","Data":"08e7ac268a718e24674c24320d8bf94329980e8f1fa5ce3b161561c65aec56cc"} Sep 09 20:56:07.792570 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:07.792537 2562 generic.go:356] "Generic (PLEG): container finished" podID="5879ea93-d3c6-4607-a910-536119156bcf" containerID="6d4f1de56066d24674b43128ab703d7fc9dddb12493175a7fb83df3cc53f1056" exitCode=0 Sep 09 20:56:07.792994 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:07.792602 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" event={"ID":"5879ea93-d3c6-4607-a910-536119156bcf","Type":"ContainerDied","Data":"6d4f1de56066d24674b43128ab703d7fc9dddb12493175a7fb83df3cc53f1056"} Sep 09 20:56:07.792994 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:07.792892 2562 scope.go:117] "RemoveContainer" containerID="6d4f1de56066d24674b43128ab703d7fc9dddb12493175a7fb83df3cc53f1056" Sep 09 20:56:08.797967 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:08.797938 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-2mxsn" event={"ID":"5879ea93-d3c6-4607-a910-536119156bcf","Type":"ContainerStarted","Data":"ee12296bbd5441e83f4ca4312da5b48597f05c13d0e6304d0cb459a0a23c5c4d"} Sep 09 20:56:37.409832 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:37.409801 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs\") pod \"network-metrics-daemon-lqd29\" (UID: \"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5\") " pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:56:37.412208 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:37.412190 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5-metrics-certs\") pod \"network-metrics-daemon-lqd29\" (UID: \"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5\") " pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:56:37.540972 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:37.540935 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kjjjq\"" Sep 09 20:56:37.545064 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:37.545042 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqd29" Sep 09 20:56:37.821126 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:37.821096 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lqd29"] Sep 09 20:56:37.827695 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:56:37.827666 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef398d96_b1a1_4b61_b0d0_1a4880fb3fb5.slice/crio-35279506b44bfcb9fb74acfbebbd8e720778d682aa36931fe7aa7776975c34ef WatchSource:0}: Error finding container 35279506b44bfcb9fb74acfbebbd8e720778d682aa36931fe7aa7776975c34ef: Status 404 returned error can't find the container with id 35279506b44bfcb9fb74acfbebbd8e720778d682aa36931fe7aa7776975c34ef Sep 09 20:56:37.905288 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:37.905232 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lqd29" event={"ID":"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5","Type":"ContainerStarted","Data":"35279506b44bfcb9fb74acfbebbd8e720778d682aa36931fe7aa7776975c34ef"} Sep 09 20:56:38.911299 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:38.911268 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lqd29" event={"ID":"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5","Type":"ContainerStarted","Data":"a326d092fb49099d7eef7c64d96a91a76ac552784c28d50a4bdf015ff0dc6a8c"} Sep 09 20:56:38.911664 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:38.911320 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lqd29" event={"ID":"ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5","Type":"ContainerStarted","Data":"6fe39ee6453ef92c9fca43da5830a6e1a632188137a6ef954c5bd8eb215cc357"} Sep 09 20:56:38.929206 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:38.929107 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lqd29" podStartSLOduration=375.079488127 podStartE2EDuration="6m15.929087358s" totalImagesPullingTime="849.599231ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:50:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:56:38.927816952 +0000 UTC m=+375.767016083" watchObservedRunningTime="2026-09-09 20:56:38.929087358 +0000 UTC m=+375.768286541" Sep 09 20:56:46.465295 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.465259 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 20:56:46.465717 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.465615 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="alertmanager" containerID="cri-o://5bf7e6bed2a55a89932bff870543d6697c9614df4e6b8a5486b82c1420d2bbaa" gracePeriod=120 Sep 09 20:56:46.465717 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.465674 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="kube-rbac-proxy-metric" containerID="cri-o://fda4e913449d0429ae06b18978eb9aeca8b90ab864f3cf757fecb9ebc492e1cb" gracePeriod=120 Sep 09 20:56:46.465844 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.465696 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="prom-label-proxy" containerID="cri-o://33a5fc73e01b03c8f949a2d5b971d2cdaad5e60198a4f85415e1c95d4d9d4b72" gracePeriod=120 Sep 09 20:56:46.465844 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.465715 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="kube-rbac-proxy-web" containerID="cri-o://bfed4077fab976a709f08932a89a7a830c93d16c277de0754b1dd6194df3e9da" gracePeriod=120 Sep 09 20:56:46.465844 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.465731 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="config-reloader" containerID="cri-o://d82126b7ee58e7dfddc12cbdc61567eb0e64c2050e53b7cf49b225ba0283290a" gracePeriod=120 Sep 09 20:56:46.465844 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.465835 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="kube-rbac-proxy" containerID="cri-o://7b027767d15768cd506179f7061873acb63959abc6d37ad97dd6e4d09057604e" gracePeriod=120 Sep 09 20:56:46.943666 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.943629 2562 generic.go:356] "Generic (PLEG): container finished" podID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerID="33a5fc73e01b03c8f949a2d5b971d2cdaad5e60198a4f85415e1c95d4d9d4b72" exitCode=0 Sep 09 20:56:46.943666 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.943658 2562 generic.go:356] "Generic (PLEG): container finished" podID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerID="fda4e913449d0429ae06b18978eb9aeca8b90ab864f3cf757fecb9ebc492e1cb" exitCode=0 Sep 09 20:56:46.943666 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.943667 2562 generic.go:356] "Generic (PLEG): container finished" podID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerID="7b027767d15768cd506179f7061873acb63959abc6d37ad97dd6e4d09057604e" exitCode=0 Sep 09 20:56:46.943666 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.943675 2562 generic.go:356] "Generic (PLEG): container finished" podID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerID="d82126b7ee58e7dfddc12cbdc61567eb0e64c2050e53b7cf49b225ba0283290a" exitCode=0 Sep 09 20:56:46.943928 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.943682 2562 generic.go:356] "Generic (PLEG): container finished" podID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerID="5bf7e6bed2a55a89932bff870543d6697c9614df4e6b8a5486b82c1420d2bbaa" exitCode=0 Sep 09 20:56:46.943928 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.943701 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f","Type":"ContainerDied","Data":"33a5fc73e01b03c8f949a2d5b971d2cdaad5e60198a4f85415e1c95d4d9d4b72"} Sep 09 20:56:46.943928 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.943763 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f","Type":"ContainerDied","Data":"fda4e913449d0429ae06b18978eb9aeca8b90ab864f3cf757fecb9ebc492e1cb"} Sep 09 20:56:46.943928 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.943789 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f","Type":"ContainerDied","Data":"7b027767d15768cd506179f7061873acb63959abc6d37ad97dd6e4d09057604e"} Sep 09 20:56:46.943928 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.943813 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f","Type":"ContainerDied","Data":"d82126b7ee58e7dfddc12cbdc61567eb0e64c2050e53b7cf49b225ba0283290a"} Sep 09 20:56:46.943928 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:46.943836 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f","Type":"ContainerDied","Data":"5bf7e6bed2a55a89932bff870543d6697c9614df4e6b8a5486b82c1420d2bbaa"} Sep 09 20:56:47.710129 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.710104 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:47.789155 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.789065 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-tls-assets\") pod \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " Sep 09 20:56:47.789338 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.789167 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-cluster-tls-config\") pod \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " Sep 09 20:56:47.789338 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.789200 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy\") pod \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " Sep 09 20:56:47.789338 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.789269 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7gxq\" (UniqueName: \"kubernetes.io/projected/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-kube-api-access-d7gxq\") pod \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " Sep 09 20:56:47.789338 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.789297 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-alertmanager-trusted-ca-bundle\") pod \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " Sep 09 20:56:47.789338 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.789325 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-config-volume\") pod \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " Sep 09 20:56:47.789603 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.789361 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-alertmanager-main-db\") pod \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " Sep 09 20:56:47.789603 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.789391 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-metrics-client-ca\") pod \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " Sep 09 20:56:47.789603 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.789422 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " Sep 09 20:56:47.789603 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.789457 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy-web\") pod \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " Sep 09 20:56:47.789603 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.789482 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-web-config\") pod \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " Sep 09 20:56:47.789603 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.789509 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-config-out\") pod \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " Sep 09 20:56:47.789603 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.789534 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-main-tls\") pod \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\" (UID: \"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f\") " Sep 09 20:56:47.790306 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.790272 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" (UID: "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 20:56:47.790745 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.790715 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" (UID: "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:56:47.790940 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.790916 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" (UID: "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 20:56:47.793771 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.792773 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" (UID: "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:56:47.794449 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.794400 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-kube-api-access-d7gxq" (OuterVolumeSpecName: "kube-api-access-d7gxq") pod "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" (UID: "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f"). InnerVolumeSpecName "kube-api-access-d7gxq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 20:56:47.794624 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.794600 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" (UID: "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:56:47.795558 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.795522 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" (UID: "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:56:47.796585 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.796551 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" (UID: "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:56:47.796970 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.796933 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-config-out" (OuterVolumeSpecName: "config-out") pod "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" (UID: "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 20:56:47.797428 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.797231 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" (UID: "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:56:47.797749 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.797722 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-config-volume" (OuterVolumeSpecName: "config-volume") pod "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" (UID: "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:56:47.800324 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.800278 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" (UID: "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:56:47.806919 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.806879 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-web-config" (OuterVolumeSpecName: "web-config") pod "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" (UID: "efc9a8cb-7dba-4ddb-9435-0d4d07d0305f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 20:56:47.890827 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.890794 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7gxq\" (UniqueName: \"kubernetes.io/projected/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-kube-api-access-d7gxq\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:56:47.890827 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.890824 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:56:47.890827 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.890834 2562 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-config-volume\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:56:47.891037 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.890843 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-alertmanager-main-db\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:56:47.891037 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.890851 2562 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-metrics-client-ca\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:56:47.891037 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.890860 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:56:47.891037 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.890868 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:56:47.891037 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.890877 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-web-config\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:56:47.891037 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.890884 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-config-out\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:56:47.891037 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.890892 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-main-tls\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:56:47.891037 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.890899 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-tls-assets\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:56:47.891037 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.890908 2562 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-cluster-tls-config\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:56:47.891037 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.890915 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 20:56:47.950044 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.950007 2562 generic.go:356] "Generic (PLEG): container finished" podID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerID="bfed4077fab976a709f08932a89a7a830c93d16c277de0754b1dd6194df3e9da" exitCode=0 Sep 09 20:56:47.950190 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.950093 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:47.950190 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.950088 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f","Type":"ContainerDied","Data":"bfed4077fab976a709f08932a89a7a830c93d16c277de0754b1dd6194df3e9da"} Sep 09 20:56:47.950290 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.950232 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"efc9a8cb-7dba-4ddb-9435-0d4d07d0305f","Type":"ContainerDied","Data":"66300587d2e9cb6cbca362b10dc150720ba826829b8b5d0d6af276c4d443d50f"} Sep 09 20:56:47.950290 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.950267 2562 scope.go:117] "RemoveContainer" containerID="33a5fc73e01b03c8f949a2d5b971d2cdaad5e60198a4f85415e1c95d4d9d4b72" Sep 09 20:56:47.958875 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.958852 2562 scope.go:117] "RemoveContainer" containerID="fda4e913449d0429ae06b18978eb9aeca8b90ab864f3cf757fecb9ebc492e1cb" Sep 09 20:56:47.966767 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.966749 2562 scope.go:117] "RemoveContainer" containerID="7b027767d15768cd506179f7061873acb63959abc6d37ad97dd6e4d09057604e" Sep 09 20:56:47.974345 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.974326 2562 scope.go:117] "RemoveContainer" containerID="bfed4077fab976a709f08932a89a7a830c93d16c277de0754b1dd6194df3e9da" Sep 09 20:56:47.981730 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.981682 2562 scope.go:117] "RemoveContainer" containerID="d82126b7ee58e7dfddc12cbdc61567eb0e64c2050e53b7cf49b225ba0283290a" Sep 09 20:56:47.982165 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.982142 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 20:56:47.989024 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.989006 2562 scope.go:117] "RemoveContainer" containerID="5bf7e6bed2a55a89932bff870543d6697c9614df4e6b8a5486b82c1420d2bbaa" Sep 09 20:56:47.995928 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:47.995910 2562 scope.go:117] "RemoveContainer" containerID="9e227a643920b60d7987d48ebafbcf3ce17d0a1fc168502e8f4eadbc61a257da" Sep 09 20:56:48.003017 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.003000 2562 scope.go:117] "RemoveContainer" containerID="33a5fc73e01b03c8f949a2d5b971d2cdaad5e60198a4f85415e1c95d4d9d4b72" Sep 09 20:56:48.003358 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:56:48.003337 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33a5fc73e01b03c8f949a2d5b971d2cdaad5e60198a4f85415e1c95d4d9d4b72\": container with ID starting with 33a5fc73e01b03c8f949a2d5b971d2cdaad5e60198a4f85415e1c95d4d9d4b72 not found: ID does not exist" containerID="33a5fc73e01b03c8f949a2d5b971d2cdaad5e60198a4f85415e1c95d4d9d4b72" Sep 09 20:56:48.003412 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.003368 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33a5fc73e01b03c8f949a2d5b971d2cdaad5e60198a4f85415e1c95d4d9d4b72"} err="failed to get container status \"33a5fc73e01b03c8f949a2d5b971d2cdaad5e60198a4f85415e1c95d4d9d4b72\": rpc error: code = NotFound desc = could not find container \"33a5fc73e01b03c8f949a2d5b971d2cdaad5e60198a4f85415e1c95d4d9d4b72\": container with ID starting with 33a5fc73e01b03c8f949a2d5b971d2cdaad5e60198a4f85415e1c95d4d9d4b72 not found: ID does not exist" Sep 09 20:56:48.003412 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.003385 2562 scope.go:117] "RemoveContainer" containerID="fda4e913449d0429ae06b18978eb9aeca8b90ab864f3cf757fecb9ebc492e1cb" Sep 09 20:56:48.003654 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:56:48.003635 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fda4e913449d0429ae06b18978eb9aeca8b90ab864f3cf757fecb9ebc492e1cb\": container with ID starting with fda4e913449d0429ae06b18978eb9aeca8b90ab864f3cf757fecb9ebc492e1cb not found: ID does not exist" containerID="fda4e913449d0429ae06b18978eb9aeca8b90ab864f3cf757fecb9ebc492e1cb" Sep 09 20:56:48.003725 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.003664 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fda4e913449d0429ae06b18978eb9aeca8b90ab864f3cf757fecb9ebc492e1cb"} err="failed to get container status \"fda4e913449d0429ae06b18978eb9aeca8b90ab864f3cf757fecb9ebc492e1cb\": rpc error: code = NotFound desc = could not find container \"fda4e913449d0429ae06b18978eb9aeca8b90ab864f3cf757fecb9ebc492e1cb\": container with ID starting with fda4e913449d0429ae06b18978eb9aeca8b90ab864f3cf757fecb9ebc492e1cb not found: ID does not exist" Sep 09 20:56:48.003725 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.003686 2562 scope.go:117] "RemoveContainer" containerID="7b027767d15768cd506179f7061873acb63959abc6d37ad97dd6e4d09057604e" Sep 09 20:56:48.003931 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:56:48.003917 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b027767d15768cd506179f7061873acb63959abc6d37ad97dd6e4d09057604e\": container with ID starting with 7b027767d15768cd506179f7061873acb63959abc6d37ad97dd6e4d09057604e not found: ID does not exist" containerID="7b027767d15768cd506179f7061873acb63959abc6d37ad97dd6e4d09057604e" Sep 09 20:56:48.003976 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.003934 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b027767d15768cd506179f7061873acb63959abc6d37ad97dd6e4d09057604e"} err="failed to get container status \"7b027767d15768cd506179f7061873acb63959abc6d37ad97dd6e4d09057604e\": rpc error: code = NotFound desc = could not find container \"7b027767d15768cd506179f7061873acb63959abc6d37ad97dd6e4d09057604e\": container with ID starting with 7b027767d15768cd506179f7061873acb63959abc6d37ad97dd6e4d09057604e not found: ID does not exist" Sep 09 20:56:48.003976 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.003945 2562 scope.go:117] "RemoveContainer" containerID="bfed4077fab976a709f08932a89a7a830c93d16c277de0754b1dd6194df3e9da" Sep 09 20:56:48.004169 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:56:48.004152 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfed4077fab976a709f08932a89a7a830c93d16c277de0754b1dd6194df3e9da\": container with ID starting with bfed4077fab976a709f08932a89a7a830c93d16c277de0754b1dd6194df3e9da not found: ID does not exist" containerID="bfed4077fab976a709f08932a89a7a830c93d16c277de0754b1dd6194df3e9da" Sep 09 20:56:48.004226 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.004176 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfed4077fab976a709f08932a89a7a830c93d16c277de0754b1dd6194df3e9da"} err="failed to get container status \"bfed4077fab976a709f08932a89a7a830c93d16c277de0754b1dd6194df3e9da\": rpc error: code = NotFound desc = could not find container \"bfed4077fab976a709f08932a89a7a830c93d16c277de0754b1dd6194df3e9da\": container with ID starting with bfed4077fab976a709f08932a89a7a830c93d16c277de0754b1dd6194df3e9da not found: ID does not exist" Sep 09 20:56:48.004226 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.004194 2562 scope.go:117] "RemoveContainer" containerID="d82126b7ee58e7dfddc12cbdc61567eb0e64c2050e53b7cf49b225ba0283290a" Sep 09 20:56:48.004416 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:56:48.004396 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d82126b7ee58e7dfddc12cbdc61567eb0e64c2050e53b7cf49b225ba0283290a\": container with ID starting with d82126b7ee58e7dfddc12cbdc61567eb0e64c2050e53b7cf49b225ba0283290a not found: ID does not exist" containerID="d82126b7ee58e7dfddc12cbdc61567eb0e64c2050e53b7cf49b225ba0283290a" Sep 09 20:56:48.004465 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.004420 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d82126b7ee58e7dfddc12cbdc61567eb0e64c2050e53b7cf49b225ba0283290a"} err="failed to get container status \"d82126b7ee58e7dfddc12cbdc61567eb0e64c2050e53b7cf49b225ba0283290a\": rpc error: code = NotFound desc = could not find container \"d82126b7ee58e7dfddc12cbdc61567eb0e64c2050e53b7cf49b225ba0283290a\": container with ID starting with d82126b7ee58e7dfddc12cbdc61567eb0e64c2050e53b7cf49b225ba0283290a not found: ID does not exist" Sep 09 20:56:48.004465 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.004440 2562 scope.go:117] "RemoveContainer" containerID="5bf7e6bed2a55a89932bff870543d6697c9614df4e6b8a5486b82c1420d2bbaa" Sep 09 20:56:48.004646 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:56:48.004633 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bf7e6bed2a55a89932bff870543d6697c9614df4e6b8a5486b82c1420d2bbaa\": container with ID starting with 5bf7e6bed2a55a89932bff870543d6697c9614df4e6b8a5486b82c1420d2bbaa not found: ID does not exist" containerID="5bf7e6bed2a55a89932bff870543d6697c9614df4e6b8a5486b82c1420d2bbaa" Sep 09 20:56:48.004688 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.004648 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf7e6bed2a55a89932bff870543d6697c9614df4e6b8a5486b82c1420d2bbaa"} err="failed to get container status \"5bf7e6bed2a55a89932bff870543d6697c9614df4e6b8a5486b82c1420d2bbaa\": rpc error: code = NotFound desc = could not find container \"5bf7e6bed2a55a89932bff870543d6697c9614df4e6b8a5486b82c1420d2bbaa\": container with ID starting with 5bf7e6bed2a55a89932bff870543d6697c9614df4e6b8a5486b82c1420d2bbaa not found: ID does not exist" Sep 09 20:56:48.004688 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.004658 2562 scope.go:117] "RemoveContainer" containerID="9e227a643920b60d7987d48ebafbcf3ce17d0a1fc168502e8f4eadbc61a257da" Sep 09 20:56:48.004860 ip-10-0-139-83 kubenswrapper[2562]: E0909 20:56:48.004844 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e227a643920b60d7987d48ebafbcf3ce17d0a1fc168502e8f4eadbc61a257da\": container with ID starting with 9e227a643920b60d7987d48ebafbcf3ce17d0a1fc168502e8f4eadbc61a257da not found: ID does not exist" containerID="9e227a643920b60d7987d48ebafbcf3ce17d0a1fc168502e8f4eadbc61a257da" Sep 09 20:56:48.004914 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.004867 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e227a643920b60d7987d48ebafbcf3ce17d0a1fc168502e8f4eadbc61a257da"} err="failed to get container status \"9e227a643920b60d7987d48ebafbcf3ce17d0a1fc168502e8f4eadbc61a257da\": rpc error: code = NotFound desc = could not find container \"9e227a643920b60d7987d48ebafbcf3ce17d0a1fc168502e8f4eadbc61a257da\": container with ID starting with 9e227a643920b60d7987d48ebafbcf3ce17d0a1fc168502e8f4eadbc61a257da not found: ID does not exist" Sep 09 20:56:48.010109 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.010090 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 20:56:48.085604 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085526 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 20:56:48.085789 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085777 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="kube-rbac-proxy-metric" Sep 09 20:56:48.085836 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085791 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="kube-rbac-proxy-metric" Sep 09 20:56:48.085836 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085800 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="kube-rbac-proxy-web" Sep 09 20:56:48.085836 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085805 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="kube-rbac-proxy-web" Sep 09 20:56:48.085836 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085812 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="kube-rbac-proxy" Sep 09 20:56:48.085836 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085818 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="kube-rbac-proxy" Sep 09 20:56:48.085836 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085825 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="prom-label-proxy" Sep 09 20:56:48.085836 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085830 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="prom-label-proxy" Sep 09 20:56:48.085836 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085839 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="config-reloader" Sep 09 20:56:48.086092 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085844 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="config-reloader" Sep 09 20:56:48.086092 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085852 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="alertmanager" Sep 09 20:56:48.086092 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085858 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="alertmanager" Sep 09 20:56:48.086092 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085864 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="372b1efb-f45d-4dae-9d3a-2070ad8099b2" containerName="registry" Sep 09 20:56:48.086092 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085869 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="372b1efb-f45d-4dae-9d3a-2070ad8099b2" containerName="registry" Sep 09 20:56:48.086092 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085877 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="init-config-reloader" Sep 09 20:56:48.086092 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085881 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="init-config-reloader" Sep 09 20:56:48.086092 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085919 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="372b1efb-f45d-4dae-9d3a-2070ad8099b2" containerName="registry" Sep 09 20:56:48.086092 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085926 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="kube-rbac-proxy-metric" Sep 09 20:56:48.086092 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085930 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="prom-label-proxy" Sep 09 20:56:48.086092 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085936 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="alertmanager" Sep 09 20:56:48.086092 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085942 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="config-reloader" Sep 09 20:56:48.086092 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085948 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="kube-rbac-proxy-web" Sep 09 20:56:48.086092 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.085954 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" containerName="kube-rbac-proxy" Sep 09 20:56:48.090885 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.090864 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.098977 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.098958 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 09 20:56:48.099099 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.099080 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-w6jhd\"" Sep 09 20:56:48.099175 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.099119 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 09 20:56:48.099175 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.099087 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 09 20:56:48.099704 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.099690 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 09 20:56:48.099804 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.099789 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 09 20:56:48.100032 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.100018 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 09 20:56:48.100081 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.100035 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 09 20:56:48.100081 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.100017 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 09 20:56:48.123399 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.123365 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 09 20:56:48.132635 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.132454 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 20:56:48.192358 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.192325 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.192358 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.192357 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-config-volume\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.192532 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.192373 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a847ee6-e620-4b7c-84c6-cc1290184610-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.192532 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.192393 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.192532 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.192478 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4a847ee6-e620-4b7c-84c6-cc1290184610-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.192532 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.192517 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82smv\" (UniqueName: \"kubernetes.io/projected/4a847ee6-e620-4b7c-84c6-cc1290184610-kube-api-access-82smv\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.192681 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.192543 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a847ee6-e620-4b7c-84c6-cc1290184610-config-out\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.192681 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.192581 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.192681 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.192597 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-web-config\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.192681 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.192622 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a847ee6-e620-4b7c-84c6-cc1290184610-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.192681 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.192652 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.192681 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.192673 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.192859 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.192718 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a847ee6-e620-4b7c-84c6-cc1290184610-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.293218 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.293164 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.293218 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.293214 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-config-volume\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.293218 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.293231 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a847ee6-e620-4b7c-84c6-cc1290184610-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.293550 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.293280 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.293550 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.293320 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4a847ee6-e620-4b7c-84c6-cc1290184610-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.293550 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.293350 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82smv\" (UniqueName: \"kubernetes.io/projected/4a847ee6-e620-4b7c-84c6-cc1290184610-kube-api-access-82smv\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.293550 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.293383 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a847ee6-e620-4b7c-84c6-cc1290184610-config-out\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.293550 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.293409 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.293550 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.293429 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-web-config\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.293550 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.293451 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a847ee6-e620-4b7c-84c6-cc1290184610-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.293550 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.293473 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.293550 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.293496 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.293550 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.293545 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a847ee6-e620-4b7c-84c6-cc1290184610-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.294201 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.293805 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4a847ee6-e620-4b7c-84c6-cc1290184610-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.294893 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.294547 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a847ee6-e620-4b7c-84c6-cc1290184610-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.294893 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.294550 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a847ee6-e620-4b7c-84c6-cc1290184610-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.296479 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.296450 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a847ee6-e620-4b7c-84c6-cc1290184610-config-out\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.296603 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.296584 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-config-volume\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.296860 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.296840 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.297344 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.297318 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a847ee6-e620-4b7c-84c6-cc1290184610-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.297446 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.297370 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.297446 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.297422 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.297627 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.297605 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.297726 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.297708 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-web-config\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.298647 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.298630 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4a847ee6-e620-4b7c-84c6-cc1290184610-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.306670 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.306652 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82smv\" (UniqueName: \"kubernetes.io/projected/4a847ee6-e620-4b7c-84c6-cc1290184610-kube-api-access-82smv\") pod \"alertmanager-main-0\" (UID: \"4a847ee6-e620-4b7c-84c6-cc1290184610\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.399917 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.399847 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 20:56:48.715798 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.715722 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 20:56:48.718673 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:56:48.718646 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a847ee6_e620_4b7c_84c6_cc1290184610.slice/crio-f594f7864b1ac913ee7cbb868c36e8c22dfaf7eb928cdc5968e6d002cc40d62e WatchSource:0}: Error finding container f594f7864b1ac913ee7cbb868c36e8c22dfaf7eb928cdc5968e6d002cc40d62e: Status 404 returned error can't find the container with id f594f7864b1ac913ee7cbb868c36e8c22dfaf7eb928cdc5968e6d002cc40d62e Sep 09 20:56:48.956127 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.956085 2562 generic.go:356] "Generic (PLEG): container finished" podID="4a847ee6-e620-4b7c-84c6-cc1290184610" containerID="79872f59832a380fd8d4e6823b0c0130cdba3bcb7334c173b439cd992ad3456e" exitCode=0 Sep 09 20:56:48.956302 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.956155 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a847ee6-e620-4b7c-84c6-cc1290184610","Type":"ContainerDied","Data":"79872f59832a380fd8d4e6823b0c0130cdba3bcb7334c173b439cd992ad3456e"} Sep 09 20:56:48.956302 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:48.956258 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a847ee6-e620-4b7c-84c6-cc1290184610","Type":"ContainerStarted","Data":"f594f7864b1ac913ee7cbb868c36e8c22dfaf7eb928cdc5968e6d002cc40d62e"} Sep 09 20:56:49.638223 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:49.638180 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efc9a8cb-7dba-4ddb-9435-0d4d07d0305f" path="/var/lib/kubelet/pods/efc9a8cb-7dba-4ddb-9435-0d4d07d0305f/volumes" Sep 09 20:56:49.963702 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:49.963613 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a847ee6-e620-4b7c-84c6-cc1290184610","Type":"ContainerStarted","Data":"344350c9fac0860d2a724703e8f9964abcf324087f9ba5e586a752a4e212ff63"} Sep 09 20:56:49.963702 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:49.963677 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a847ee6-e620-4b7c-84c6-cc1290184610","Type":"ContainerStarted","Data":"e7ffd2eec03b2b8c034be4482bb8805e4bda77253b032a6a26662e5000f5a6ad"} Sep 09 20:56:49.963702 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:49.963703 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a847ee6-e620-4b7c-84c6-cc1290184610","Type":"ContainerStarted","Data":"bfbda6f831632807face1eaff5210b1334b689d69140bf3828b3c980d94ce348"} Sep 09 20:56:49.964160 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:49.963725 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a847ee6-e620-4b7c-84c6-cc1290184610","Type":"ContainerStarted","Data":"6766ad4114f12485ff170b85d92a0999b2248f9b4812b6b3f3cbb8a1a4375693"} Sep 09 20:56:49.964160 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:49.963748 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a847ee6-e620-4b7c-84c6-cc1290184610","Type":"ContainerStarted","Data":"1b8505470c585d66ca946f82f10f05e5ee37c995d059fa0e1e7e9f1c2af606da"} Sep 09 20:56:49.964160 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:49.963772 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4a847ee6-e620-4b7c-84c6-cc1290184610","Type":"ContainerStarted","Data":"b8f823a79918f89a00caa92b06051737349e4f09b2ee9da143456b3957506f4f"} Sep 09 20:56:50.593397 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.593339 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.575615679 podStartE2EDuration="2.593321436s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.705757ms" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:56:50.015923253 +0000 UTC m=+386.855122382" watchObservedRunningTime="2026-09-09 20:56:50.593321436 +0000 UTC m=+387.432520590" Sep 09 20:56:50.593683 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.593669 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-d7c6bc989-qxn8g"] Sep 09 20:56:50.598002 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.597987 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.601804 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.601783 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 09 20:56:50.603710 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.603691 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 09 20:56:50.604564 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.604318 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 09 20:56:50.604564 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.604549 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 09 20:56:50.605089 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.605069 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 09 20:56:50.606395 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.606381 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-rjmcg\"" Sep 09 20:56:50.610177 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.610161 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 09 20:56:50.643475 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.643443 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-d7c6bc989-qxn8g"] Sep 09 20:56:50.716405 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.716373 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5a6791c7-e21c-4e3f-986f-2251bb52e447-federate-client-tls\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.716405 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.716407 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5a6791c7-e21c-4e3f-986f-2251bb52e447-telemeter-client-tls\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.716580 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.716442 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a6791c7-e21c-4e3f-986f-2251bb52e447-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.716580 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.716463 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc4pj\" (UniqueName: \"kubernetes.io/projected/5a6791c7-e21c-4e3f-986f-2251bb52e447-kube-api-access-gc4pj\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.716580 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.716527 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a6791c7-e21c-4e3f-986f-2251bb52e447-serving-certs-ca-bundle\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.716580 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.716573 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a6791c7-e21c-4e3f-986f-2251bb52e447-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.716699 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.716602 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a6791c7-e21c-4e3f-986f-2251bb52e447-metrics-client-ca\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.716699 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.716624 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5a6791c7-e21c-4e3f-986f-2251bb52e447-secret-telemeter-client\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.817358 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.817323 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5a6791c7-e21c-4e3f-986f-2251bb52e447-federate-client-tls\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.817358 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.817356 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5a6791c7-e21c-4e3f-986f-2251bb52e447-telemeter-client-tls\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.817524 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.817380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a6791c7-e21c-4e3f-986f-2251bb52e447-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.817524 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.817400 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc4pj\" (UniqueName: \"kubernetes.io/projected/5a6791c7-e21c-4e3f-986f-2251bb52e447-kube-api-access-gc4pj\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.817524 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.817453 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a6791c7-e21c-4e3f-986f-2251bb52e447-serving-certs-ca-bundle\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.817524 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.817513 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a6791c7-e21c-4e3f-986f-2251bb52e447-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.817701 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.817555 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a6791c7-e21c-4e3f-986f-2251bb52e447-metrics-client-ca\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.817701 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.817591 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5a6791c7-e21c-4e3f-986f-2251bb52e447-secret-telemeter-client\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.818424 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.818361 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a6791c7-e21c-4e3f-986f-2251bb52e447-metrics-client-ca\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.818424 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.818397 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a6791c7-e21c-4e3f-986f-2251bb52e447-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.819105 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.819081 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a6791c7-e21c-4e3f-986f-2251bb52e447-serving-certs-ca-bundle\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.820492 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.820463 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5a6791c7-e21c-4e3f-986f-2251bb52e447-telemeter-client-tls\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.820690 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.820670 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5a6791c7-e21c-4e3f-986f-2251bb52e447-secret-telemeter-client\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.820735 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.820670 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a6791c7-e21c-4e3f-986f-2251bb52e447-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.820770 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.820755 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5a6791c7-e21c-4e3f-986f-2251bb52e447-federate-client-tls\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.829812 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.829792 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc4pj\" (UniqueName: \"kubernetes.io/projected/5a6791c7-e21c-4e3f-986f-2251bb52e447-kube-api-access-gc4pj\") pod \"telemeter-client-d7c6bc989-qxn8g\" (UID: \"5a6791c7-e21c-4e3f-986f-2251bb52e447\") " pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:50.908065 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:50.907989 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" Sep 09 20:56:51.205565 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:51.205350 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-d7c6bc989-qxn8g"] Sep 09 20:56:51.208021 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:56:51.207999 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a6791c7_e21c_4e3f_986f_2251bb52e447.slice/crio-fd2098f9eb956c8abdc8c42e148acb52619dbc010188c4f8cad41bf7c569201a WatchSource:0}: Error finding container fd2098f9eb956c8abdc8c42e148acb52619dbc010188c4f8cad41bf7c569201a: Status 404 returned error can't find the container with id fd2098f9eb956c8abdc8c42e148acb52619dbc010188c4f8cad41bf7c569201a Sep 09 20:56:51.973718 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:51.973678 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" event={"ID":"5a6791c7-e21c-4e3f-986f-2251bb52e447","Type":"ContainerStarted","Data":"fd2098f9eb956c8abdc8c42e148acb52619dbc010188c4f8cad41bf7c569201a"} Sep 09 20:56:52.979457 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:52.979429 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" event={"ID":"5a6791c7-e21c-4e3f-986f-2251bb52e447","Type":"ContainerStarted","Data":"f91c39230ca7ea3d5e68b9fffdea64ace4a7dd68ffefbcb5def399a126978990"} Sep 09 20:56:52.979756 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:52.979483 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" event={"ID":"5a6791c7-e21c-4e3f-986f-2251bb52e447","Type":"ContainerStarted","Data":"a068811d4e40b0f643eee8b0b433ca78459f10aab2521bc7e469fa305da6d650"} Sep 09 20:56:53.985431 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:56:53.985391 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" event={"ID":"5a6791c7-e21c-4e3f-986f-2251bb52e447","Type":"ContainerStarted","Data":"ec14377353009841d1e4458b066a4a8d681bdae31295408009b7f08c68445a63"} Sep 09 20:59:22.490205 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:22.490096 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-d7c6bc989-qxn8g" podStartSLOduration=150.917948153 podStartE2EDuration="2m32.490077306s" totalImagesPullingTime="1.572129153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:56:54.042466901 +0000 UTC m=+390.881666032" watchObservedRunningTime="2026-09-09 20:59:22.490077306 +0000 UTC m=+539.329276437" Sep 09 20:59:22.491535 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:22.491511 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tw6lg"] Sep 09 20:59:22.494835 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:22.494816 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tw6lg" Sep 09 20:59:22.500549 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:22.500527 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 09 20:59:22.500955 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:22.500931 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-5rskl\"" Sep 09 20:59:22.500955 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:22.500949 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 09 20:59:22.535687 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:22.535660 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tw6lg"] Sep 09 20:59:22.627830 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:22.627793 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e4eaa52-5bed-442a-a1bc-a11350004d86-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tw6lg\" (UID: \"2e4eaa52-5bed-442a-a1bc-a11350004d86\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tw6lg" Sep 09 20:59:22.627830 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:22.627829 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9gpn\" (UniqueName: \"kubernetes.io/projected/2e4eaa52-5bed-442a-a1bc-a11350004d86-kube-api-access-z9gpn\") pod \"cert-manager-webhook-6486ffd997-tw6lg\" (UID: \"2e4eaa52-5bed-442a-a1bc-a11350004d86\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tw6lg" Sep 09 20:59:22.728996 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:22.728963 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e4eaa52-5bed-442a-a1bc-a11350004d86-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tw6lg\" (UID: \"2e4eaa52-5bed-442a-a1bc-a11350004d86\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tw6lg" Sep 09 20:59:22.728996 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:22.728996 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9gpn\" (UniqueName: \"kubernetes.io/projected/2e4eaa52-5bed-442a-a1bc-a11350004d86-kube-api-access-z9gpn\") pod \"cert-manager-webhook-6486ffd997-tw6lg\" (UID: \"2e4eaa52-5bed-442a-a1bc-a11350004d86\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tw6lg" Sep 09 20:59:22.771120 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:22.771047 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9gpn\" (UniqueName: \"kubernetes.io/projected/2e4eaa52-5bed-442a-a1bc-a11350004d86-kube-api-access-z9gpn\") pod \"cert-manager-webhook-6486ffd997-tw6lg\" (UID: \"2e4eaa52-5bed-442a-a1bc-a11350004d86\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tw6lg" Sep 09 20:59:22.782788 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:22.782754 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e4eaa52-5bed-442a-a1bc-a11350004d86-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tw6lg\" (UID: \"2e4eaa52-5bed-442a-a1bc-a11350004d86\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tw6lg" Sep 09 20:59:22.803751 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:22.803725 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tw6lg" Sep 09 20:59:23.142280 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:23.142256 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tw6lg"] Sep 09 20:59:23.145132 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:59:23.145108 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e4eaa52_5bed_442a_a1bc_a11350004d86.slice/crio-3dcbbedb5ac12c13601f59ed8375ca0cfbe789fb9fb6d0707251a1ab99c00bf7 WatchSource:0}: Error finding container 3dcbbedb5ac12c13601f59ed8375ca0cfbe789fb9fb6d0707251a1ab99c00bf7: Status 404 returned error can't find the container with id 3dcbbedb5ac12c13601f59ed8375ca0cfbe789fb9fb6d0707251a1ab99c00bf7 Sep 09 20:59:23.540807 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:23.540769 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tw6lg" event={"ID":"2e4eaa52-5bed-442a-a1bc-a11350004d86","Type":"ContainerStarted","Data":"3dcbbedb5ac12c13601f59ed8375ca0cfbe789fb9fb6d0707251a1ab99c00bf7"} Sep 09 20:59:26.556461 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:26.556412 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tw6lg" event={"ID":"2e4eaa52-5bed-442a-a1bc-a11350004d86","Type":"ContainerStarted","Data":"e13576a2fe31eb12cdcfabb5b4944d98ae7dbc0b8bc96606e4fe283453fa53f1"} Sep 09 20:59:26.556840 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:26.556513 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tw6lg" Sep 09 20:59:26.589796 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:26.589743 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-tw6lg" podStartSLOduration=1.271387378 podStartE2EDuration="4.58972707s" totalImagesPullingTime="3.318339692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:59:26.588533076 +0000 UTC m=+543.427732206" watchObservedRunningTime="2026-09-09 20:59:26.58972707 +0000 UTC m=+543.428926199" Sep 09 20:59:32.562561 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:32.562525 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tw6lg" Sep 09 20:59:33.562408 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:33.562373 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c75ft"] Sep 09 20:59:33.565618 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:33.565602 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c75ft" Sep 09 20:59:33.568053 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:33.568030 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-9vtj4\"" Sep 09 20:59:33.581760 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:33.581739 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c75ft"] Sep 09 20:59:33.621322 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:33.621292 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82gjd\" (UniqueName: \"kubernetes.io/projected/7290af42-79a7-4895-97d0-1119a1c6a924-kube-api-access-82gjd\") pod \"cert-manager-55b4d954d5-c75ft\" (UID: \"7290af42-79a7-4895-97d0-1119a1c6a924\") " pod="cert-manager/cert-manager-55b4d954d5-c75ft" Sep 09 20:59:33.621474 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:33.621345 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7290af42-79a7-4895-97d0-1119a1c6a924-bound-sa-token\") pod \"cert-manager-55b4d954d5-c75ft\" (UID: \"7290af42-79a7-4895-97d0-1119a1c6a924\") " pod="cert-manager/cert-manager-55b4d954d5-c75ft" Sep 09 20:59:33.722285 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:33.722248 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82gjd\" (UniqueName: \"kubernetes.io/projected/7290af42-79a7-4895-97d0-1119a1c6a924-kube-api-access-82gjd\") pod \"cert-manager-55b4d954d5-c75ft\" (UID: \"7290af42-79a7-4895-97d0-1119a1c6a924\") " pod="cert-manager/cert-manager-55b4d954d5-c75ft" Sep 09 20:59:33.722478 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:33.722343 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7290af42-79a7-4895-97d0-1119a1c6a924-bound-sa-token\") pod \"cert-manager-55b4d954d5-c75ft\" (UID: \"7290af42-79a7-4895-97d0-1119a1c6a924\") " pod="cert-manager/cert-manager-55b4d954d5-c75ft" Sep 09 20:59:33.750525 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:33.750493 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82gjd\" (UniqueName: \"kubernetes.io/projected/7290af42-79a7-4895-97d0-1119a1c6a924-kube-api-access-82gjd\") pod \"cert-manager-55b4d954d5-c75ft\" (UID: \"7290af42-79a7-4895-97d0-1119a1c6a924\") " pod="cert-manager/cert-manager-55b4d954d5-c75ft" Sep 09 20:59:33.754911 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:33.754888 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7290af42-79a7-4895-97d0-1119a1c6a924-bound-sa-token\") pod \"cert-manager-55b4d954d5-c75ft\" (UID: \"7290af42-79a7-4895-97d0-1119a1c6a924\") " pod="cert-manager/cert-manager-55b4d954d5-c75ft" Sep 09 20:59:33.874333 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:33.874256 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c75ft" Sep 09 20:59:34.153608 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:34.153583 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c75ft"] Sep 09 20:59:34.156195 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:59:34.156173 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7290af42_79a7_4895_97d0_1119a1c6a924.slice/crio-897b336b13086611b36412435cd54fb75874d849a94422bc3b7562db28d975e2 WatchSource:0}: Error finding container 897b336b13086611b36412435cd54fb75874d849a94422bc3b7562db28d975e2: Status 404 returned error can't find the container with id 897b336b13086611b36412435cd54fb75874d849a94422bc3b7562db28d975e2 Sep 09 20:59:34.590330 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:34.590293 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c75ft" event={"ID":"7290af42-79a7-4895-97d0-1119a1c6a924","Type":"ContainerStarted","Data":"d60138878811d4fb7fab6c71dfb8ff9cb1fea9870a2b8294815797d385b4cfd2"} Sep 09 20:59:34.590746 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:34.590339 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c75ft" event={"ID":"7290af42-79a7-4895-97d0-1119a1c6a924","Type":"ContainerStarted","Data":"897b336b13086611b36412435cd54fb75874d849a94422bc3b7562db28d975e2"} Sep 09 20:59:34.621845 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:34.621791 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-c75ft" podStartSLOduration=1.6217758039999999 podStartE2EDuration="1.621775804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:59:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:59:34.619876833 +0000 UTC m=+551.459075962" watchObservedRunningTime="2026-09-09 20:59:34.621775804 +0000 UTC m=+551.460974935" Sep 09 20:59:54.109316 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.109282 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x"] Sep 09 20:59:54.112617 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.112596 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.116599 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.116542 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 09 20:59:54.116759 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.116620 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-v47mq\"" Sep 09 20:59:54.116759 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.116646 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 09 20:59:54.116759 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.116750 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 09 20:59:54.116920 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.116778 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 09 20:59:54.128899 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.128873 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x"] Sep 09 20:59:54.184498 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.184410 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0178f901-e8d9-42b6-b07a-662dadc9ace3-manifests\") pod \"opendatahub-operator-controller-manager-cf9547dff-glg6x\" (UID: \"0178f901-e8d9-42b6-b07a-662dadc9ace3\") " pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.184498 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.184459 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clmkb\" (UniqueName: \"kubernetes.io/projected/0178f901-e8d9-42b6-b07a-662dadc9ace3-kube-api-access-clmkb\") pod \"opendatahub-operator-controller-manager-cf9547dff-glg6x\" (UID: \"0178f901-e8d9-42b6-b07a-662dadc9ace3\") " pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.184679 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.184533 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0178f901-e8d9-42b6-b07a-662dadc9ace3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-cf9547dff-glg6x\" (UID: \"0178f901-e8d9-42b6-b07a-662dadc9ace3\") " pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.184679 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.184564 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0178f901-e8d9-42b6-b07a-662dadc9ace3-webhook-cert\") pod \"opendatahub-operator-controller-manager-cf9547dff-glg6x\" (UID: \"0178f901-e8d9-42b6-b07a-662dadc9ace3\") " pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.184679 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.184609 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0178f901-e8d9-42b6-b07a-662dadc9ace3-tmp\") pod \"opendatahub-operator-controller-manager-cf9547dff-glg6x\" (UID: \"0178f901-e8d9-42b6-b07a-662dadc9ace3\") " pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.285250 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.285214 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0178f901-e8d9-42b6-b07a-662dadc9ace3-manifests\") pod \"opendatahub-operator-controller-manager-cf9547dff-glg6x\" (UID: \"0178f901-e8d9-42b6-b07a-662dadc9ace3\") " pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.285416 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.285286 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clmkb\" (UniqueName: \"kubernetes.io/projected/0178f901-e8d9-42b6-b07a-662dadc9ace3-kube-api-access-clmkb\") pod \"opendatahub-operator-controller-manager-cf9547dff-glg6x\" (UID: \"0178f901-e8d9-42b6-b07a-662dadc9ace3\") " pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.285416 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.285320 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0178f901-e8d9-42b6-b07a-662dadc9ace3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-cf9547dff-glg6x\" (UID: \"0178f901-e8d9-42b6-b07a-662dadc9ace3\") " pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.285416 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.285336 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0178f901-e8d9-42b6-b07a-662dadc9ace3-webhook-cert\") pod \"opendatahub-operator-controller-manager-cf9547dff-glg6x\" (UID: \"0178f901-e8d9-42b6-b07a-662dadc9ace3\") " pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.285416 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.285370 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0178f901-e8d9-42b6-b07a-662dadc9ace3-tmp\") pod \"opendatahub-operator-controller-manager-cf9547dff-glg6x\" (UID: \"0178f901-e8d9-42b6-b07a-662dadc9ace3\") " pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.285670 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.285647 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0178f901-e8d9-42b6-b07a-662dadc9ace3-manifests\") pod \"opendatahub-operator-controller-manager-cf9547dff-glg6x\" (UID: \"0178f901-e8d9-42b6-b07a-662dadc9ace3\") " pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.287769 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.287736 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0178f901-e8d9-42b6-b07a-662dadc9ace3-tmp\") pod \"opendatahub-operator-controller-manager-cf9547dff-glg6x\" (UID: \"0178f901-e8d9-42b6-b07a-662dadc9ace3\") " pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.287914 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.287900 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0178f901-e8d9-42b6-b07a-662dadc9ace3-webhook-cert\") pod \"opendatahub-operator-controller-manager-cf9547dff-glg6x\" (UID: \"0178f901-e8d9-42b6-b07a-662dadc9ace3\") " pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.288067 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.288045 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0178f901-e8d9-42b6-b07a-662dadc9ace3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-cf9547dff-glg6x\" (UID: \"0178f901-e8d9-42b6-b07a-662dadc9ace3\") " pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.295296 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.295213 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clmkb\" (UniqueName: \"kubernetes.io/projected/0178f901-e8d9-42b6-b07a-662dadc9ace3-kube-api-access-clmkb\") pod \"opendatahub-operator-controller-manager-cf9547dff-glg6x\" (UID: \"0178f901-e8d9-42b6-b07a-662dadc9ace3\") " pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.423222 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.423186 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 20:59:54.719746 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:54.719634 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x"] Sep 09 20:59:54.726037 ip-10-0-139-83 kubenswrapper[2562]: W0909 20:59:54.726011 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0178f901_e8d9_42b6_b07a_662dadc9ace3.slice/crio-f8d2a067e9f4c76b3f2c369e68b25eef9e1ae85f930e76c841bd25d7f61aa800 WatchSource:0}: Error finding container f8d2a067e9f4c76b3f2c369e68b25eef9e1ae85f930e76c841bd25d7f61aa800: Status 404 returned error can't find the container with id f8d2a067e9f4c76b3f2c369e68b25eef9e1ae85f930e76c841bd25d7f61aa800 Sep 09 20:59:55.669693 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:55.669620 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" event={"ID":"0178f901-e8d9-42b6-b07a-662dadc9ace3","Type":"ContainerStarted","Data":"f8d2a067e9f4c76b3f2c369e68b25eef9e1ae85f930e76c841bd25d7f61aa800"} Sep 09 20:59:57.680737 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:57.680698 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" event={"ID":"0178f901-e8d9-42b6-b07a-662dadc9ace3","Type":"ContainerStarted","Data":"7f71dc0968d4b2bae66505e00d537e5483aa24d69d213f34dc0795a21b9bc04f"} Sep 09 20:59:59.690605 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:59.690574 2562 generic.go:356] "Generic (PLEG): container finished" podID="0178f901-e8d9-42b6-b07a-662dadc9ace3" containerID="7f71dc0968d4b2bae66505e00d537e5483aa24d69d213f34dc0795a21b9bc04f" exitCode=0 Sep 09 20:59:59.690987 ip-10-0-139-83 kubenswrapper[2562]: I0909 20:59:59.690641 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" event={"ID":"0178f901-e8d9-42b6-b07a-662dadc9ace3","Type":"ContainerDied","Data":"7f71dc0968d4b2bae66505e00d537e5483aa24d69d213f34dc0795a21b9bc04f"} Sep 09 21:00:00.697509 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:00.697475 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" event={"ID":"0178f901-e8d9-42b6-b07a-662dadc9ace3","Type":"ContainerStarted","Data":"8a72b66d7773df214b8593011873cb57de6b52359be67cbd4f65f9f35e9a51b3"} Sep 09 21:00:00.697903 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:00.697690 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 21:00:00.722170 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:00.722118 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" podStartSLOduration=2.355754106 podStartE2EDuration="6.722096155s" totalImagesPullingTime="3.334331496s" totalInitContainerRuntime="1.032010553s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:59:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:00.720387406 +0000 UTC m=+577.559586551" watchObservedRunningTime="2026-09-09 21:00:00.722096155 +0000 UTC m=+577.561295311" Sep 09 21:00:11.703945 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:11.703916 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-cf9547dff-glg6x" Sep 09 21:00:13.818931 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:13.818897 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56"] Sep 09 21:00:13.827012 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:13.826987 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56" Sep 09 21:00:13.841215 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:13.841195 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 09 21:00:13.845274 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:13.845258 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 09 21:00:13.846120 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:13.846107 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-t6d28\"" Sep 09 21:00:13.846177 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:13.846164 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 09 21:00:13.852407 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:13.852388 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 09 21:00:13.954388 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:13.954352 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba348cb1-cd13-4c92-854e-b059cd5612c6-tls-certs\") pod \"kube-auth-proxy-55f8b7786b-7rk56\" (UID: \"ba348cb1-cd13-4c92-854e-b059cd5612c6\") " pod="openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56" Sep 09 21:00:13.954557 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:13.954430 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t2sf\" (UniqueName: \"kubernetes.io/projected/ba348cb1-cd13-4c92-854e-b059cd5612c6-kube-api-access-8t2sf\") pod \"kube-auth-proxy-55f8b7786b-7rk56\" (UID: \"ba348cb1-cd13-4c92-854e-b059cd5612c6\") " pod="openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56" Sep 09 21:00:13.954557 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:13.954451 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba348cb1-cd13-4c92-854e-b059cd5612c6-tmp\") pod \"kube-auth-proxy-55f8b7786b-7rk56\" (UID: \"ba348cb1-cd13-4c92-854e-b059cd5612c6\") " pod="openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56" Sep 09 21:00:13.959272 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:13.959231 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56"] Sep 09 21:00:14.055360 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:14.055316 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t2sf\" (UniqueName: \"kubernetes.io/projected/ba348cb1-cd13-4c92-854e-b059cd5612c6-kube-api-access-8t2sf\") pod \"kube-auth-proxy-55f8b7786b-7rk56\" (UID: \"ba348cb1-cd13-4c92-854e-b059cd5612c6\") " pod="openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56" Sep 09 21:00:14.055360 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:14.055364 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba348cb1-cd13-4c92-854e-b059cd5612c6-tmp\") pod \"kube-auth-proxy-55f8b7786b-7rk56\" (UID: \"ba348cb1-cd13-4c92-854e-b059cd5612c6\") " pod="openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56" Sep 09 21:00:14.055607 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:14.055431 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba348cb1-cd13-4c92-854e-b059cd5612c6-tls-certs\") pod \"kube-auth-proxy-55f8b7786b-7rk56\" (UID: \"ba348cb1-cd13-4c92-854e-b059cd5612c6\") " pod="openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56" Sep 09 21:00:14.058371 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:14.058342 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba348cb1-cd13-4c92-854e-b059cd5612c6-tmp\") pod \"kube-auth-proxy-55f8b7786b-7rk56\" (UID: \"ba348cb1-cd13-4c92-854e-b059cd5612c6\") " pod="openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56" Sep 09 21:00:14.058493 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:14.058435 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba348cb1-cd13-4c92-854e-b059cd5612c6-tls-certs\") pod \"kube-auth-proxy-55f8b7786b-7rk56\" (UID: \"ba348cb1-cd13-4c92-854e-b059cd5612c6\") " pod="openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56" Sep 09 21:00:14.093063 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:14.092994 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t2sf\" (UniqueName: \"kubernetes.io/projected/ba348cb1-cd13-4c92-854e-b059cd5612c6-kube-api-access-8t2sf\") pod \"kube-auth-proxy-55f8b7786b-7rk56\" (UID: \"ba348cb1-cd13-4c92-854e-b059cd5612c6\") " pod="openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56" Sep 09 21:00:14.136880 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:14.136849 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56" Sep 09 21:00:14.415730 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:14.415707 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56"] Sep 09 21:00:14.417533 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:00:14.417501 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba348cb1_cd13_4c92_854e_b059cd5612c6.slice/crio-56abb083d01b495b86320e3671adce37c044dd7fbde74ad90bcdd4bc03a4762e WatchSource:0}: Error finding container 56abb083d01b495b86320e3671adce37c044dd7fbde74ad90bcdd4bc03a4762e: Status 404 returned error can't find the container with id 56abb083d01b495b86320e3671adce37c044dd7fbde74ad90bcdd4bc03a4762e Sep 09 21:00:14.760423 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:14.760376 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56" event={"ID":"ba348cb1-cd13-4c92-854e-b059cd5612c6","Type":"ContainerStarted","Data":"56abb083d01b495b86320e3671adce37c044dd7fbde74ad90bcdd4bc03a4762e"} Sep 09 21:00:17.778762 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:17.778725 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56" event={"ID":"ba348cb1-cd13-4c92-854e-b059cd5612c6","Type":"ContainerStarted","Data":"adafd0414dfe490fd0434a6d7cf22fcabe5656c83e1040eb625816a56ed89221"} Sep 09 21:00:17.798726 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:17.798678 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-55f8b7786b-7rk56" podStartSLOduration=1.970335444 podStartE2EDuration="4.798661973s" totalImagesPullingTime="2.828326529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:17.796412776 +0000 UTC m=+594.635611919" watchObservedRunningTime="2026-09-09 21:00:17.798661973 +0000 UTC m=+594.637861102" Sep 09 21:00:20.592338 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.592303 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4"] Sep 09 21:00:20.595712 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.595690 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.598204 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.598183 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 09 21:00:20.598378 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.598359 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-wkxss\"" Sep 09 21:00:20.606394 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.606374 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4"] Sep 09 21:00:20.716009 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.715976 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.716172 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.716013 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.716172 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.716032 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.716172 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.716057 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.716172 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.716113 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.716172 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.716140 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9lg9\" (UniqueName: \"kubernetes.io/projected/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-kube-api-access-x9lg9\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.716172 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.716162 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.716485 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.716257 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.716485 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.716286 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.817561 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.817531 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.817561 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.817563 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.817788 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.817581 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.817788 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.817599 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.817788 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.817618 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9lg9\" (UniqueName: \"kubernetes.io/projected/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-kube-api-access-x9lg9\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.817788 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.817641 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.817788 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.817689 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.818030 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.817849 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.818030 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.817944 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.818251 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.818184 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.818365 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.818260 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.818365 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.818345 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.818487 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.818366 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.819010 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.818986 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.820277 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.820232 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.820390 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.820348 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.825959 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.825938 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9lg9\" (UniqueName: \"kubernetes.io/projected/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-kube-api-access-x9lg9\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.827401 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.827380 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d64d0526-7ac1-4363-8fa9-b10cb0e9d334-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4\" (UID: \"d64d0526-7ac1-4363-8fa9-b10cb0e9d334\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:20.906666 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:20.906574 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:21.189664 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:21.189578 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4"] Sep 09 21:00:21.196998 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:00:21.196968 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd64d0526_7ac1_4363_8fa9_b10cb0e9d334.slice/crio-5a46e0134d210c04d9074655bc18faef9501f1257ed61b905fc26e16746074a8 WatchSource:0}: Error finding container 5a46e0134d210c04d9074655bc18faef9501f1257ed61b905fc26e16746074a8: Status 404 returned error can't find the container with id 5a46e0134d210c04d9074655bc18faef9501f1257ed61b905fc26e16746074a8 Sep 09 21:00:21.797584 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:21.797544 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" event={"ID":"d64d0526-7ac1-4363-8fa9-b10cb0e9d334","Type":"ContainerStarted","Data":"5a46e0134d210c04d9074655bc18faef9501f1257ed61b905fc26e16746074a8"} Sep 09 21:00:24.154757 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:24.154719 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"} Sep 09 21:00:24.155044 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:24.154799 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"} Sep 09 21:00:24.155044 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:24.154829 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"} Sep 09 21:00:24.422842 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:24.422758 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ckv66"] Sep 09 21:00:24.426368 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:24.426351 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-ckv66" Sep 09 21:00:24.429327 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:24.429306 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-nvq56\"" Sep 09 21:00:24.429531 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:24.429517 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 09 21:00:24.442311 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:24.442283 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ckv66"] Sep 09 21:00:24.453910 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:24.453878 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59qpb\" (UniqueName: \"kubernetes.io/projected/8d78d46f-4740-404e-984e-f647488edb65-kube-api-access-59qpb\") pod \"odh-model-controller-7998c67dc4-ckv66\" (UID: \"8d78d46f-4740-404e-984e-f647488edb65\") " pod="opendatahub/odh-model-controller-7998c67dc4-ckv66" Sep 09 21:00:24.454054 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:24.453947 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d78d46f-4740-404e-984e-f647488edb65-cert\") pod \"odh-model-controller-7998c67dc4-ckv66\" (UID: \"8d78d46f-4740-404e-984e-f647488edb65\") " pod="opendatahub/odh-model-controller-7998c67dc4-ckv66" Sep 09 21:00:24.555346 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:24.555313 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59qpb\" (UniqueName: \"kubernetes.io/projected/8d78d46f-4740-404e-984e-f647488edb65-kube-api-access-59qpb\") pod \"odh-model-controller-7998c67dc4-ckv66\" (UID: \"8d78d46f-4740-404e-984e-f647488edb65\") " pod="opendatahub/odh-model-controller-7998c67dc4-ckv66" Sep 09 21:00:24.555506 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:24.555356 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d78d46f-4740-404e-984e-f647488edb65-cert\") pod \"odh-model-controller-7998c67dc4-ckv66\" (UID: \"8d78d46f-4740-404e-984e-f647488edb65\") " pod="opendatahub/odh-model-controller-7998c67dc4-ckv66" Sep 09 21:00:24.555506 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:00:24.555480 2562 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 09 21:00:24.555641 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:00:24.555538 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d78d46f-4740-404e-984e-f647488edb65-cert podName:8d78d46f-4740-404e-984e-f647488edb65 nodeName:}" failed. No retries permitted until 2026-09-09 21:00:25.055521442 +0000 UTC m=+601.894720551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8d78d46f-4740-404e-984e-f647488edb65-cert") pod "odh-model-controller-7998c67dc4-ckv66" (UID: "8d78d46f-4740-404e-984e-f647488edb65") : secret "odh-model-controller-webhook-cert" not found Sep 09 21:00:24.583396 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:24.583366 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59qpb\" (UniqueName: \"kubernetes.io/projected/8d78d46f-4740-404e-984e-f647488edb65-kube-api-access-59qpb\") pod \"odh-model-controller-7998c67dc4-ckv66\" (UID: \"8d78d46f-4740-404e-984e-f647488edb65\") " pod="opendatahub/odh-model-controller-7998c67dc4-ckv66" Sep 09 21:00:24.816447 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:24.816399 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" event={"ID":"d64d0526-7ac1-4363-8fa9-b10cb0e9d334","Type":"ContainerStarted","Data":"2d6241d58e3574957f503f321499b24035dc06cf17c83d095d39856d02d95ba6"} Sep 09 21:00:24.841559 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:24.841505 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" podStartSLOduration=1.8862802090000002 podStartE2EDuration="4.841485376s" totalImagesPullingTime="2.955205167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:24.839697066 +0000 UTC m=+601.678896196" watchObservedRunningTime="2026-09-09 21:00:24.841485376 +0000 UTC m=+601.680684518" Sep 09 21:00:24.907591 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:24.907558 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:25.060408 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.060373 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d78d46f-4740-404e-984e-f647488edb65-cert\") pod \"odh-model-controller-7998c67dc4-ckv66\" (UID: \"8d78d46f-4740-404e-984e-f647488edb65\") " pod="opendatahub/odh-model-controller-7998c67dc4-ckv66" Sep 09 21:00:25.062876 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.062857 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d78d46f-4740-404e-984e-f647488edb65-cert\") pod \"odh-model-controller-7998c67dc4-ckv66\" (UID: \"8d78d46f-4740-404e-984e-f647488edb65\") " pod="opendatahub/odh-model-controller-7998c67dc4-ckv66" Sep 09 21:00:25.094726 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.094659 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8trts"] Sep 09 21:00:25.097733 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.097719 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-8trts" Sep 09 21:00:25.099976 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.099955 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-kb5ff\"" Sep 09 21:00:25.100098 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.099955 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 09 21:00:25.111920 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.111893 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8trts"] Sep 09 21:00:25.161075 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.161046 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b65b188-5a8e-4307-9d22-4ffd70c6b7c6-tls-certs\") pod \"model-serving-api-54d89bd5fc-8trts\" (UID: \"3b65b188-5a8e-4307-9d22-4ffd70c6b7c6\") " pod="opendatahub/model-serving-api-54d89bd5fc-8trts" Sep 09 21:00:25.161492 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.161174 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnzxf\" (UniqueName: \"kubernetes.io/projected/3b65b188-5a8e-4307-9d22-4ffd70c6b7c6-kube-api-access-gnzxf\") pod \"model-serving-api-54d89bd5fc-8trts\" (UID: \"3b65b188-5a8e-4307-9d22-4ffd70c6b7c6\") " pod="opendatahub/model-serving-api-54d89bd5fc-8trts" Sep 09 21:00:25.262046 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.262011 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnzxf\" (UniqueName: \"kubernetes.io/projected/3b65b188-5a8e-4307-9d22-4ffd70c6b7c6-kube-api-access-gnzxf\") pod \"model-serving-api-54d89bd5fc-8trts\" (UID: \"3b65b188-5a8e-4307-9d22-4ffd70c6b7c6\") " pod="opendatahub/model-serving-api-54d89bd5fc-8trts" Sep 09 21:00:25.262046 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.262059 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b65b188-5a8e-4307-9d22-4ffd70c6b7c6-tls-certs\") pod \"model-serving-api-54d89bd5fc-8trts\" (UID: \"3b65b188-5a8e-4307-9d22-4ffd70c6b7c6\") " pod="opendatahub/model-serving-api-54d89bd5fc-8trts" Sep 09 21:00:25.262283 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:00:25.262153 2562 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Sep 09 21:00:25.262283 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:00:25.262212 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b65b188-5a8e-4307-9d22-4ffd70c6b7c6-tls-certs podName:3b65b188-5a8e-4307-9d22-4ffd70c6b7c6 nodeName:}" failed. No retries permitted until 2026-09-09 21:00:25.762196183 +0000 UTC m=+602.601395291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/3b65b188-5a8e-4307-9d22-4ffd70c6b7c6-tls-certs") pod "model-serving-api-54d89bd5fc-8trts" (UID: "3b65b188-5a8e-4307-9d22-4ffd70c6b7c6") : secret "model-serving-api-tls" not found Sep 09 21:00:25.283147 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.283118 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnzxf\" (UniqueName: \"kubernetes.io/projected/3b65b188-5a8e-4307-9d22-4ffd70c6b7c6-kube-api-access-gnzxf\") pod \"model-serving-api-54d89bd5fc-8trts\" (UID: \"3b65b188-5a8e-4307-9d22-4ffd70c6b7c6\") " pod="opendatahub/model-serving-api-54d89bd5fc-8trts" Sep 09 21:00:25.337010 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.336979 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-ckv66" Sep 09 21:00:25.612789 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.612626 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ckv66"] Sep 09 21:00:25.615139 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:00:25.615113 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d78d46f_4740_404e_984e_f647488edb65.slice/crio-f2a5d5c8561e38508f15780f73e055866b5809e49305a41506d42a5bd3f67eaa WatchSource:0}: Error finding container f2a5d5c8561e38508f15780f73e055866b5809e49305a41506d42a5bd3f67eaa: Status 404 returned error can't find the container with id f2a5d5c8561e38508f15780f73e055866b5809e49305a41506d42a5bd3f67eaa Sep 09 21:00:25.767323 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.767292 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b65b188-5a8e-4307-9d22-4ffd70c6b7c6-tls-certs\") pod \"model-serving-api-54d89bd5fc-8trts\" (UID: \"3b65b188-5a8e-4307-9d22-4ffd70c6b7c6\") " pod="opendatahub/model-serving-api-54d89bd5fc-8trts" Sep 09 21:00:25.769866 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.769840 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b65b188-5a8e-4307-9d22-4ffd70c6b7c6-tls-certs\") pod \"model-serving-api-54d89bd5fc-8trts\" (UID: \"3b65b188-5a8e-4307-9d22-4ffd70c6b7c6\") " pod="opendatahub/model-serving-api-54d89bd5fc-8trts" Sep 09 21:00:25.821749 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.821716 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-ckv66" event={"ID":"8d78d46f-4740-404e-984e-f647488edb65","Type":"ContainerStarted","Data":"f2a5d5c8561e38508f15780f73e055866b5809e49305a41506d42a5bd3f67eaa"} Sep 09 21:00:25.912222 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:25.912153 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:26.007651 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:26.007625 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-8trts" Sep 09 21:00:26.292889 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:26.292860 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8trts"] Sep 09 21:00:26.294732 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:00:26.294707 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b65b188_5a8e_4307_9d22_4ffd70c6b7c6.slice/crio-45293df839743de73e49321b853076beb962d6cf276e250a17e4bdd8b1360fdd WatchSource:0}: Error finding container 45293df839743de73e49321b853076beb962d6cf276e250a17e4bdd8b1360fdd: Status 404 returned error can't find the container with id 45293df839743de73e49321b853076beb962d6cf276e250a17e4bdd8b1360fdd Sep 09 21:00:26.827830 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:26.827789 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-8trts" event={"ID":"3b65b188-5a8e-4307-9d22-4ffd70c6b7c6","Type":"ContainerStarted","Data":"45293df839743de73e49321b853076beb962d6cf276e250a17e4bdd8b1360fdd"} Sep 09 21:00:26.828003 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:26.827971 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:26.828882 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:26.828865 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4" Sep 09 21:00:29.846319 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:29.846284 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-ckv66" event={"ID":"8d78d46f-4740-404e-984e-f647488edb65","Type":"ContainerStarted","Data":"f0ea168e3347f3bbf60bb113bee59d6baf6f222a31b994947579f9f6b7725b42"} Sep 09 21:00:29.846795 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:29.846474 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-ckv66" Sep 09 21:00:29.847817 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:29.847788 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-8trts" event={"ID":"3b65b188-5a8e-4307-9d22-4ffd70c6b7c6","Type":"ContainerStarted","Data":"50634556fe7757eb49deed8de8f77be378ee51874eb14ab93ab3601613787b92"} Sep 09 21:00:29.847941 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:29.847925 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-8trts" Sep 09 21:00:29.865329 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:29.865282 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-ckv66" podStartSLOduration=2.268589737 podStartE2EDuration="5.865267969s" totalImagesPullingTime="3.596678232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:29.864060615 +0000 UTC m=+606.703259746" watchObservedRunningTime="2026-09-09 21:00:29.865267969 +0000 UTC m=+606.704467095" Sep 09 21:00:29.883064 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:29.883020 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-8trts" podStartSLOduration=2.020130243 podStartE2EDuration="4.883004497s" totalImagesPullingTime="2.862874254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:29.882265687 +0000 UTC m=+606.721464821" watchObservedRunningTime="2026-09-09 21:00:29.883004497 +0000 UTC m=+606.722203627" Sep 09 21:00:30.971165 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:30.971130 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g"] Sep 09 21:00:30.974588 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:30.974573 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:30.977109 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:30.977088 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 09 21:00:30.977109 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:30.977102 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 09 21:00:30.977455 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:30.977440 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-pndpf\"" Sep 09 21:00:30.977949 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:30.977936 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 09 21:00:30.992404 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:30.992378 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g"] Sep 09 21:00:31.014722 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.014696 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.014867 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.014731 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wltvz\" (UniqueName: \"kubernetes.io/projected/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-kube-api-access-wltvz\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.014867 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.014756 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.014867 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.014830 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.014996 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.014881 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.014996 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.014896 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.014996 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.014933 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.115808 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.115773 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.115969 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.115823 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.115969 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.115849 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wltvz\" (UniqueName: \"kubernetes.io/projected/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-kube-api-access-wltvz\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.115969 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.115870 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.115969 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.115898 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.115969 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.115939 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.115969 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.115961 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.116899 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.116870 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.118466 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.118432 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.118466 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.118462 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.118646 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.118589 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.118646 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.118600 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.125074 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.125052 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.125317 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.125300 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wltvz\" (UniqueName: \"kubernetes.io/projected/2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c-kube-api-access-wltvz\") pod \"istiod-openshift-gateway-5677ccb49b-s7g9g\" (UID: \"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.283603 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.283519 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:31.565522 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.565496 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g"] Sep 09 21:00:31.567388 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:00:31.567360 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d14b124_b5c8_4b6e_a8a1_f6e20a26db4c.slice/crio-628bcd5a678bcfbb797e0085c54667e7d13e25a8d6b7f34607984c6362b45296 WatchSource:0}: Error finding container 628bcd5a678bcfbb797e0085c54667e7d13e25a8d6b7f34607984c6362b45296: Status 404 returned error can't find the container with id 628bcd5a678bcfbb797e0085c54667e7d13e25a8d6b7f34607984c6362b45296 Sep 09 21:00:31.569557 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.569538 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:00:31.857773 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:31.857698 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" event={"ID":"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c","Type":"ContainerStarted","Data":"628bcd5a678bcfbb797e0085c54667e7d13e25a8d6b7f34607984c6362b45296"} Sep 09 21:00:33.843550 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:33.843512 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"} Sep 09 21:00:33.843922 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:33.843591 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"} Sep 09 21:00:34.874966 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:34.874930 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" event={"ID":"2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c","Type":"ContainerStarted","Data":"036ac83141c3a7830a47334d733ef9df27e87d6a9a7ee867db0f1a0cde68e5c4"} Sep 09 21:00:34.875346 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:34.874990 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:34.876665 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:34.876641 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" Sep 09 21:00:34.900128 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:34.900087 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s7g9g" podStartSLOduration=2.626461143 podStartE2EDuration="4.900071775s" totalImagesPullingTime="2.273610632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:34.89815947 +0000 UTC m=+611.737358617" watchObservedRunningTime="2026-09-09 21:00:34.900071775 +0000 UTC m=+611.739270906" Sep 09 21:00:40.854563 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:40.854524 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-ckv66" Sep 09 21:00:40.856725 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:00:40.856701 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-8trts" Sep 09 21:01:43.668203 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:43.668173 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7"] Sep 09 21:01:43.675861 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:43.675834 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:01:43.678232 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:43.678207 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 09 21:01:43.678378 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:43.678306 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 09 21:01:43.678428 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:43.678393 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rdnlv\"" Sep 09 21:01:43.683813 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:43.683793 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7"] Sep 09 21:01:43.834778 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:43.834743 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25pwj\" (UniqueName: \"kubernetes.io/projected/505fc21f-a9af-4107-9f53-4fda9d4105a6-kube-api-access-25pwj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mb6b7\" (UID: \"505fc21f-a9af-4107-9f53-4fda9d4105a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:01:43.834951 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:43.834845 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/505fc21f-a9af-4107-9f53-4fda9d4105a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mb6b7\" (UID: \"505fc21f-a9af-4107-9f53-4fda9d4105a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:01:43.935975 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:43.935888 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25pwj\" (UniqueName: \"kubernetes.io/projected/505fc21f-a9af-4107-9f53-4fda9d4105a6-kube-api-access-25pwj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mb6b7\" (UID: \"505fc21f-a9af-4107-9f53-4fda9d4105a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:01:43.935975 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:43.935945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/505fc21f-a9af-4107-9f53-4fda9d4105a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mb6b7\" (UID: \"505fc21f-a9af-4107-9f53-4fda9d4105a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:01:43.936355 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:43.936337 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/505fc21f-a9af-4107-9f53-4fda9d4105a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mb6b7\" (UID: \"505fc21f-a9af-4107-9f53-4fda9d4105a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:01:43.952066 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:43.952041 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25pwj\" (UniqueName: \"kubernetes.io/projected/505fc21f-a9af-4107-9f53-4fda9d4105a6-kube-api-access-25pwj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mb6b7\" (UID: \"505fc21f-a9af-4107-9f53-4fda9d4105a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:01:43.988245 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:43.988215 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:01:44.280356 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:44.280325 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7"] Sep 09 21:01:44.281481 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:01:44.281454 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod505fc21f_a9af_4107_9f53_4fda9d4105a6.slice/crio-3f7a5eff0fafa500fe54f1fc2712d887be7a5d361984dc5242e5a00be73e6edf WatchSource:0}: Error finding container 3f7a5eff0fafa500fe54f1fc2712d887be7a5d361984dc5242e5a00be73e6edf: Status 404 returned error can't find the container with id 3f7a5eff0fafa500fe54f1fc2712d887be7a5d361984dc5242e5a00be73e6edf Sep 09 21:01:45.187738 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:45.185666 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" event={"ID":"505fc21f-a9af-4107-9f53-4fda9d4105a6","Type":"ContainerStarted","Data":"3f7a5eff0fafa500fe54f1fc2712d887be7a5d361984dc5242e5a00be73e6edf"} Sep 09 21:01:49.212332 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:49.212296 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" event={"ID":"505fc21f-a9af-4107-9f53-4fda9d4105a6","Type":"ContainerStarted","Data":"2dd8a889b2e8099579110f1564189819543d76f69e6e52c16748872fe58e10d1"} Sep 09 21:01:49.212810 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:49.212483 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:01:49.250407 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:49.250349 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" podStartSLOduration=2.135789517 podStartE2EDuration="6.250332442s" totalImagesPullingTime="4.114542925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:49.24952654 +0000 UTC m=+686.088725669" watchObservedRunningTime="2026-09-09 21:01:49.250332442 +0000 UTC m=+686.089531583" Sep 09 21:01:50.814530 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:50.814499 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t"] Sep 09 21:01:50.818281 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:50.818252 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:01:50.821510 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:50.821486 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-cl6ww\"" Sep 09 21:01:50.831733 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:50.831709 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t"] Sep 09 21:01:51.002424 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:51.002394 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h59t6\" (UniqueName: \"kubernetes.io/projected/144cb606-150e-4e30-9bdc-ed71985b53b8-kube-api-access-h59t6\") pod \"limitador-operator-controller-manager-74495c69f8-7pn2t\" (UID: \"144cb606-150e-4e30-9bdc-ed71985b53b8\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:01:51.103289 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:51.103171 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h59t6\" (UniqueName: \"kubernetes.io/projected/144cb606-150e-4e30-9bdc-ed71985b53b8-kube-api-access-h59t6\") pod \"limitador-operator-controller-manager-74495c69f8-7pn2t\" (UID: \"144cb606-150e-4e30-9bdc-ed71985b53b8\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:01:51.118924 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:51.118891 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h59t6\" (UniqueName: \"kubernetes.io/projected/144cb606-150e-4e30-9bdc-ed71985b53b8-kube-api-access-h59t6\") pod \"limitador-operator-controller-manager-74495c69f8-7pn2t\" (UID: \"144cb606-150e-4e30-9bdc-ed71985b53b8\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:01:51.128665 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:51.128644 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:01:51.412644 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:51.412621 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t"] Sep 09 21:01:51.414693 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:01:51.414667 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod144cb606_150e_4e30_9bdc_ed71985b53b8.slice/crio-930e405dc239ef4d6ac8dcca0a56bec45151d2f7ee6ea637d345f016287d3f16 WatchSource:0}: Error finding container 930e405dc239ef4d6ac8dcca0a56bec45151d2f7ee6ea637d345f016287d3f16: Status 404 returned error can't find the container with id 930e405dc239ef4d6ac8dcca0a56bec45151d2f7ee6ea637d345f016287d3f16 Sep 09 21:01:52.227137 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:52.227098 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" event={"ID":"144cb606-150e-4e30-9bdc-ed71985b53b8","Type":"ContainerStarted","Data":"930e405dc239ef4d6ac8dcca0a56bec45151d2f7ee6ea637d345f016287d3f16"} Sep 09 21:01:53.233838 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:53.233751 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" event={"ID":"144cb606-150e-4e30-9bdc-ed71985b53b8","Type":"ContainerStarted","Data":"45588744310ded1fecb2314677d195de74a0a55a7d5fb8f46bc940a3850a3449"} Sep 09 21:01:53.234188 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:53.233835 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:01:53.261308 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:01:53.261263 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" podStartSLOduration=1.704682413 podStartE2EDuration="3.261223043s" totalImagesPullingTime="1.55654063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:53.258648306 +0000 UTC m=+690.097847436" watchObservedRunningTime="2026-09-09 21:01:53.261223043 +0000 UTC m=+690.100422174" Sep 09 21:02:00.219341 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:00.219307 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:02:02.357281 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.357232 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7"] Sep 09 21:02:02.357689 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.357444 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" podUID="505fc21f-a9af-4107-9f53-4fda9d4105a6" containerName="manager" containerID="cri-o://2dd8a889b2e8099579110f1564189819543d76f69e6e52c16748872fe58e10d1" gracePeriod=2 Sep 09 21:02:02.362723 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.362699 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7"] Sep 09 21:02:02.374618 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.374592 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t"] Sep 09 21:02:02.374847 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.374813 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" podUID="144cb606-150e-4e30-9bdc-ed71985b53b8" containerName="manager" containerID="cri-o://45588744310ded1fecb2314677d195de74a0a55a7d5fb8f46bc940a3850a3449" gracePeriod=2 Sep 09 21:02:02.376945 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.376920 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:02:02.385807 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.385784 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t"] Sep 09 21:02:02.398127 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:02.398075 2562 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"144cb606-150e-4e30-9bdc-ed71985b53b8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-7pn2t\": pods \"limitador-operator-controller-manager-74495c69f8-7pn2t\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:02:02.400411 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.400385 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6"] Sep 09 21:02:02.400942 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:02.400914 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-mb6b7\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="505fc21f-a9af-4107-9f53-4fda9d4105a6" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:02:02.401111 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.401089 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="144cb606-150e-4e30-9bdc-ed71985b53b8" containerName="manager" Sep 09 21:02:02.401175 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.401116 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="144cb606-150e-4e30-9bdc-ed71985b53b8" containerName="manager" Sep 09 21:02:02.401175 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.401139 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="505fc21f-a9af-4107-9f53-4fda9d4105a6" containerName="manager" Sep 09 21:02:02.401175 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.401149 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="505fc21f-a9af-4107-9f53-4fda9d4105a6" containerName="manager" Sep 09 21:02:02.401350 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.401261 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="505fc21f-a9af-4107-9f53-4fda9d4105a6" containerName="manager" Sep 09 21:02:02.401350 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.401275 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="144cb606-150e-4e30-9bdc-ed71985b53b8" containerName="manager" Sep 09 21:02:02.403458 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:02.403428 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7pn2t\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="144cb606-150e-4e30-9bdc-ed71985b53b8" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:02:02.404677 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.404655 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" Sep 09 21:02:02.405439 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:02.405416 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-mb6b7\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="505fc21f-a9af-4107-9f53-4fda9d4105a6" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:02:02.407226 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:02.407202 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7pn2t\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="144cb606-150e-4e30-9bdc-ed71985b53b8" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:02:02.409627 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:02.409580 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-mb6b7\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="505fc21f-a9af-4107-9f53-4fda9d4105a6" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:02:02.411403 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.411384 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-vp9dp"] Sep 09 21:02:02.414726 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.414709 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-vp9dp" Sep 09 21:02:02.430215 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.430190 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6"] Sep 09 21:02:02.468151 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.468119 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-vp9dp"] Sep 09 21:02:02.487781 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.487752 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4079c714-876a-4382-b05f-139114b4bfe7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-d4ms6\" (UID: \"4079c714-876a-4382-b05f-139114b4bfe7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" Sep 09 21:02:02.487924 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.487901 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk9nv\" (UniqueName: \"kubernetes.io/projected/733a6c42-1ad7-4ca8-bb1c-44f80595149e-kube-api-access-rk9nv\") pod \"limitador-operator-controller-manager-74495c69f8-vp9dp\" (UID: \"733a6c42-1ad7-4ca8-bb1c-44f80595149e\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-vp9dp" Sep 09 21:02:02.488009 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.487960 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlzvn\" (UniqueName: \"kubernetes.io/projected/4079c714-876a-4382-b05f-139114b4bfe7-kube-api-access-hlzvn\") pod \"kuadrant-operator-controller-manager-76f8f686bf-d4ms6\" (UID: \"4079c714-876a-4382-b05f-139114b4bfe7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" Sep 09 21:02:02.513941 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:02.513849 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7pn2t\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="144cb606-150e-4e30-9bdc-ed71985b53b8" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:02:02.516088 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:02.516049 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-mb6b7\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="505fc21f-a9af-4107-9f53-4fda9d4105a6" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:02:02.588947 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.588914 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk9nv\" (UniqueName: \"kubernetes.io/projected/733a6c42-1ad7-4ca8-bb1c-44f80595149e-kube-api-access-rk9nv\") pod \"limitador-operator-controller-manager-74495c69f8-vp9dp\" (UID: \"733a6c42-1ad7-4ca8-bb1c-44f80595149e\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-vp9dp" Sep 09 21:02:02.589120 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.588969 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlzvn\" (UniqueName: \"kubernetes.io/projected/4079c714-876a-4382-b05f-139114b4bfe7-kube-api-access-hlzvn\") pod \"kuadrant-operator-controller-manager-76f8f686bf-d4ms6\" (UID: \"4079c714-876a-4382-b05f-139114b4bfe7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" Sep 09 21:02:02.589184 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.589157 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4079c714-876a-4382-b05f-139114b4bfe7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-d4ms6\" (UID: \"4079c714-876a-4382-b05f-139114b4bfe7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" Sep 09 21:02:02.589526 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.589506 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4079c714-876a-4382-b05f-139114b4bfe7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-d4ms6\" (UID: \"4079c714-876a-4382-b05f-139114b4bfe7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" Sep 09 21:02:02.600857 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.600834 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk9nv\" (UniqueName: \"kubernetes.io/projected/733a6c42-1ad7-4ca8-bb1c-44f80595149e-kube-api-access-rk9nv\") pod \"limitador-operator-controller-manager-74495c69f8-vp9dp\" (UID: \"733a6c42-1ad7-4ca8-bb1c-44f80595149e\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-vp9dp" Sep 09 21:02:02.614827 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.614808 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlzvn\" (UniqueName: \"kubernetes.io/projected/4079c714-876a-4382-b05f-139114b4bfe7-kube-api-access-hlzvn\") pod \"kuadrant-operator-controller-manager-76f8f686bf-d4ms6\" (UID: \"4079c714-876a-4382-b05f-139114b4bfe7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" Sep 09 21:02:02.618359 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.618338 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:02:02.620672 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:02.620647 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7pn2t\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="144cb606-150e-4e30-9bdc-ed71985b53b8" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:02:02.620913 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.620900 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:02:02.623717 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:02.623697 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-mb6b7\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="505fc21f-a9af-4107-9f53-4fda9d4105a6" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:02:02.626470 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:02.626441 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7pn2t\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="144cb606-150e-4e30-9bdc-ed71985b53b8" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:02:02.628283 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:02.628262 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-mb6b7\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="505fc21f-a9af-4107-9f53-4fda9d4105a6" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:02:02.689949 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.689928 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25pwj\" (UniqueName: \"kubernetes.io/projected/505fc21f-a9af-4107-9f53-4fda9d4105a6-kube-api-access-25pwj\") pod \"505fc21f-a9af-4107-9f53-4fda9d4105a6\" (UID: \"505fc21f-a9af-4107-9f53-4fda9d4105a6\") " Sep 09 21:02:02.690053 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.689978 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h59t6\" (UniqueName: \"kubernetes.io/projected/144cb606-150e-4e30-9bdc-ed71985b53b8-kube-api-access-h59t6\") pod \"144cb606-150e-4e30-9bdc-ed71985b53b8\" (UID: \"144cb606-150e-4e30-9bdc-ed71985b53b8\") " Sep 09 21:02:02.690053 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.690019 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/505fc21f-a9af-4107-9f53-4fda9d4105a6-extensions-socket-volume\") pod \"505fc21f-a9af-4107-9f53-4fda9d4105a6\" (UID: \"505fc21f-a9af-4107-9f53-4fda9d4105a6\") " Sep 09 21:02:02.690612 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.690588 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/505fc21f-a9af-4107-9f53-4fda9d4105a6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "505fc21f-a9af-4107-9f53-4fda9d4105a6" (UID: "505fc21f-a9af-4107-9f53-4fda9d4105a6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:02:02.692181 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.692160 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/144cb606-150e-4e30-9bdc-ed71985b53b8-kube-api-access-h59t6" (OuterVolumeSpecName: "kube-api-access-h59t6") pod "144cb606-150e-4e30-9bdc-ed71985b53b8" (UID: "144cb606-150e-4e30-9bdc-ed71985b53b8"). InnerVolumeSpecName "kube-api-access-h59t6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:02:02.692311 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.692175 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/505fc21f-a9af-4107-9f53-4fda9d4105a6-kube-api-access-25pwj" (OuterVolumeSpecName: "kube-api-access-25pwj") pod "505fc21f-a9af-4107-9f53-4fda9d4105a6" (UID: "505fc21f-a9af-4107-9f53-4fda9d4105a6"). InnerVolumeSpecName "kube-api-access-25pwj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:02:02.782058 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.782034 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" Sep 09 21:02:02.790040 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.790015 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-vp9dp" Sep 09 21:02:02.791477 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.791453 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25pwj\" (UniqueName: \"kubernetes.io/projected/505fc21f-a9af-4107-9f53-4fda9d4105a6-kube-api-access-25pwj\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 21:02:02.791541 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.791483 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h59t6\" (UniqueName: \"kubernetes.io/projected/144cb606-150e-4e30-9bdc-ed71985b53b8-kube-api-access-h59t6\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 21:02:02.791541 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:02.791497 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/505fc21f-a9af-4107-9f53-4fda9d4105a6-extensions-socket-volume\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 21:02:03.074034 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.074007 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-vp9dp"] Sep 09 21:02:03.075680 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.075655 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6"] Sep 09 21:02:03.076227 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:02:03.076202 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod733a6c42_1ad7_4ca8_bb1c_44f80595149e.slice/crio-8277b62d2fa80cb4e39714dfdf2b96f98935b6518f618caa557de5bbe582685c WatchSource:0}: Error finding container 8277b62d2fa80cb4e39714dfdf2b96f98935b6518f618caa557de5bbe582685c: Status 404 returned error can't find the container with id 8277b62d2fa80cb4e39714dfdf2b96f98935b6518f618caa557de5bbe582685c Sep 09 21:02:03.077222 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:02:03.077195 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4079c714_876a_4382_b05f_139114b4bfe7.slice/crio-451083daa63d71fa862b2fd29d8df89813c85e59e4e4ab89d148cc67099983e3 WatchSource:0}: Error finding container 451083daa63d71fa862b2fd29d8df89813c85e59e4e4ab89d148cc67099983e3: Status 404 returned error can't find the container with id 451083daa63d71fa862b2fd29d8df89813c85e59e4e4ab89d148cc67099983e3 Sep 09 21:02:03.288697 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.288595 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" event={"ID":"4079c714-876a-4382-b05f-139114b4bfe7","Type":"ContainerStarted","Data":"32f8f31629b5bda968ebb752b11dfcfee8ab912220791428933673924b94af75"} Sep 09 21:02:03.288697 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.288662 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" event={"ID":"4079c714-876a-4382-b05f-139114b4bfe7","Type":"ContainerStarted","Data":"451083daa63d71fa862b2fd29d8df89813c85e59e4e4ab89d148cc67099983e3"} Sep 09 21:02:03.288951 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.288726 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" Sep 09 21:02:03.290711 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.290683 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-vp9dp" event={"ID":"733a6c42-1ad7-4ca8-bb1c-44f80595149e","Type":"ContainerStarted","Data":"c050184c1e9919ae5a0df0ffa5fb8dddfcaf506d2d58ffc4665e1272458c3470"} Sep 09 21:02:03.290867 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.290729 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-vp9dp" event={"ID":"733a6c42-1ad7-4ca8-bb1c-44f80595149e","Type":"ContainerStarted","Data":"8277b62d2fa80cb4e39714dfdf2b96f98935b6518f618caa557de5bbe582685c"} Sep 09 21:02:03.290867 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:03.290725 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7pn2t\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="144cb606-150e-4e30-9bdc-ed71985b53b8" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:02:03.290867 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.290757 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-vp9dp" Sep 09 21:02:03.292368 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.292337 2562 generic.go:356] "Generic (PLEG): container finished" podID="505fc21f-a9af-4107-9f53-4fda9d4105a6" containerID="2dd8a889b2e8099579110f1564189819543d76f69e6e52c16748872fe58e10d1" exitCode=0 Sep 09 21:02:03.292479 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.292384 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:02:03.292479 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.292437 2562 scope.go:117] "RemoveContainer" containerID="2dd8a889b2e8099579110f1564189819543d76f69e6e52c16748872fe58e10d1" Sep 09 21:02:03.293770 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:03.293750 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-mb6b7\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="505fc21f-a9af-4107-9f53-4fda9d4105a6" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:02:03.294095 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.294069 2562 generic.go:356] "Generic (PLEG): container finished" podID="144cb606-150e-4e30-9bdc-ed71985b53b8" containerID="45588744310ded1fecb2314677d195de74a0a55a7d5fb8f46bc940a3850a3449" exitCode=0 Sep 09 21:02:03.294453 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.294101 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:02:03.301730 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.301713 2562 scope.go:117] "RemoveContainer" containerID="2dd8a889b2e8099579110f1564189819543d76f69e6e52c16748872fe58e10d1" Sep 09 21:02:03.302089 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:03.302044 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dd8a889b2e8099579110f1564189819543d76f69e6e52c16748872fe58e10d1\": container with ID starting with 2dd8a889b2e8099579110f1564189819543d76f69e6e52c16748872fe58e10d1 not found: ID does not exist" containerID="2dd8a889b2e8099579110f1564189819543d76f69e6e52c16748872fe58e10d1" Sep 09 21:02:03.302167 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.302099 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dd8a889b2e8099579110f1564189819543d76f69e6e52c16748872fe58e10d1"} err="failed to get container status \"2dd8a889b2e8099579110f1564189819543d76f69e6e52c16748872fe58e10d1\": rpc error: code = NotFound desc = could not find container \"2dd8a889b2e8099579110f1564189819543d76f69e6e52c16748872fe58e10d1\": container with ID starting with 2dd8a889b2e8099579110f1564189819543d76f69e6e52c16748872fe58e10d1 not found: ID does not exist" Sep 09 21:02:03.302167 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.302126 2562 scope.go:117] "RemoveContainer" containerID="45588744310ded1fecb2314677d195de74a0a55a7d5fb8f46bc940a3850a3449" Sep 09 21:02:03.310057 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.310037 2562 scope.go:117] "RemoveContainer" containerID="45588744310ded1fecb2314677d195de74a0a55a7d5fb8f46bc940a3850a3449" Sep 09 21:02:03.310433 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:03.310397 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45588744310ded1fecb2314677d195de74a0a55a7d5fb8f46bc940a3850a3449\": container with ID starting with 45588744310ded1fecb2314677d195de74a0a55a7d5fb8f46bc940a3850a3449 not found: ID does not exist" containerID="45588744310ded1fecb2314677d195de74a0a55a7d5fb8f46bc940a3850a3449" Sep 09 21:02:03.310540 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.310437 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45588744310ded1fecb2314677d195de74a0a55a7d5fb8f46bc940a3850a3449"} err="failed to get container status \"45588744310ded1fecb2314677d195de74a0a55a7d5fb8f46bc940a3850a3449\": rpc error: code = NotFound desc = could not find container \"45588744310ded1fecb2314677d195de74a0a55a7d5fb8f46bc940a3850a3449\": container with ID starting with 45588744310ded1fecb2314677d195de74a0a55a7d5fb8f46bc940a3850a3449 not found: ID does not exist" Sep 09 21:02:03.323935 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.323889 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" podStartSLOduration=1.323875201 podStartE2EDuration="1.323875201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:02:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:03.322444304 +0000 UTC m=+700.161643446" watchObservedRunningTime="2026-09-09 21:02:03.323875201 +0000 UTC m=+700.163074344" Sep 09 21:02:03.345952 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.345906 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-vp9dp" podStartSLOduration=1.345891718 podStartE2EDuration="1.345891718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:02:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:03.344086131 +0000 UTC m=+700.183285281" watchObservedRunningTime="2026-09-09 21:02:03.345891718 +0000 UTC m=+700.185090847" Sep 09 21:02:03.348496 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:03.348472 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7pn2t\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="144cb606-150e-4e30-9bdc-ed71985b53b8" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:02:03.352841 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:03.352818 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-mb6b7\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="505fc21f-a9af-4107-9f53-4fda9d4105a6" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:02:03.640461 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:03.640364 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7pn2t\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="144cb606-150e-4e30-9bdc-ed71985b53b8" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7pn2t" Sep 09 21:02:03.642084 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:03.642055 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-mb6b7\" is forbidden: User \"system:node:ip-10-0-139-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-83.ec2.internal' and this object" podUID="505fc21f-a9af-4107-9f53-4fda9d4105a6" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mb6b7" Sep 09 21:02:03.644003 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.643979 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="144cb606-150e-4e30-9bdc-ed71985b53b8" path="/var/lib/kubelet/pods/144cb606-150e-4e30-9bdc-ed71985b53b8/volumes" Sep 09 21:02:03.644404 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.644390 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="505fc21f-a9af-4107-9f53-4fda9d4105a6" path="/var/lib/kubelet/pods/505fc21f-a9af-4107-9f53-4fda9d4105a6/volumes" Sep 09 21:02:03.723918 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.723887 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lz82c"] Sep 09 21:02:03.728803 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.728783 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lz82c" Sep 09 21:02:03.743040 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.743019 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lz82c"] Sep 09 21:02:03.798325 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.798296 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2mkh\" (UniqueName: \"kubernetes.io/projected/b9afc124-9af3-4e6f-9c70-d9f911d1c7de-kube-api-access-k2mkh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lz82c\" (UID: \"b9afc124-9af3-4e6f-9c70-d9f911d1c7de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lz82c" Sep 09 21:02:03.798479 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.798346 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9afc124-9af3-4e6f-9c70-d9f911d1c7de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lz82c\" (UID: \"b9afc124-9af3-4e6f-9c70-d9f911d1c7de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lz82c" Sep 09 21:02:03.899616 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.899537 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9afc124-9af3-4e6f-9c70-d9f911d1c7de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lz82c\" (UID: \"b9afc124-9af3-4e6f-9c70-d9f911d1c7de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lz82c" Sep 09 21:02:03.899793 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.899632 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2mkh\" (UniqueName: \"kubernetes.io/projected/b9afc124-9af3-4e6f-9c70-d9f911d1c7de-kube-api-access-k2mkh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lz82c\" (UID: \"b9afc124-9af3-4e6f-9c70-d9f911d1c7de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lz82c" Sep 09 21:02:03.899930 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.899908 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9afc124-9af3-4e6f-9c70-d9f911d1c7de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lz82c\" (UID: \"b9afc124-9af3-4e6f-9c70-d9f911d1c7de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lz82c" Sep 09 21:02:03.909438 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:03.909409 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2mkh\" (UniqueName: \"kubernetes.io/projected/b9afc124-9af3-4e6f-9c70-d9f911d1c7de-kube-api-access-k2mkh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lz82c\" (UID: \"b9afc124-9af3-4e6f-9c70-d9f911d1c7de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lz82c" Sep 09 21:02:04.039509 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:04.039476 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lz82c" Sep 09 21:02:04.325701 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:04.325678 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lz82c"] Sep 09 21:02:04.327061 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:02:04.327034 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9afc124_9af3_4e6f_9c70_d9f911d1c7de.slice/crio-c3217133aa0045b8acef11849a115c2cf5c0cd7f88ecac5fa5dab3089627a02f WatchSource:0}: Error finding container c3217133aa0045b8acef11849a115c2cf5c0cd7f88ecac5fa5dab3089627a02f: Status 404 returned error can't find the container with id c3217133aa0045b8acef11849a115c2cf5c0cd7f88ecac5fa5dab3089627a02f Sep 09 21:02:05.307570 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:05.307534 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lz82c" event={"ID":"b9afc124-9af3-4e6f-9c70-d9f911d1c7de","Type":"ContainerStarted","Data":"76cd7181fc0b9535e83997ffe01fc55369b9d621d70577fc267cbe895090c2ef"} Sep 09 21:02:05.308050 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:05.307582 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lz82c" event={"ID":"b9afc124-9af3-4e6f-9c70-d9f911d1c7de","Type":"ContainerStarted","Data":"c3217133aa0045b8acef11849a115c2cf5c0cd7f88ecac5fa5dab3089627a02f"} Sep 09 21:02:05.308050 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:05.307660 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lz82c" Sep 09 21:02:05.330428 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:05.330381 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lz82c" podStartSLOduration=2.330367684 podStartE2EDuration="2.330367684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:02:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:05.328363942 +0000 UTC m=+702.167563072" watchObservedRunningTime="2026-09-09 21:02:05.330367684 +0000 UTC m=+702.169566813" Sep 09 21:02:14.303498 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:14.303466 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" Sep 09 21:02:14.303898 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:14.303569 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-vp9dp" Sep 09 21:02:16.314665 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:16.314635 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lz82c" Sep 09 21:02:16.377315 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:16.377286 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6"] Sep 09 21:02:16.377533 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:16.377503 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" podUID="4079c714-876a-4382-b05f-139114b4bfe7" containerName="manager" containerID="cri-o://32f8f31629b5bda968ebb752b11dfcfee8ab912220791428933673924b94af75" gracePeriod=10 Sep 09 21:02:16.627906 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:16.627880 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" Sep 09 21:02:16.714142 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:16.714117 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlzvn\" (UniqueName: \"kubernetes.io/projected/4079c714-876a-4382-b05f-139114b4bfe7-kube-api-access-hlzvn\") pod \"4079c714-876a-4382-b05f-139114b4bfe7\" (UID: \"4079c714-876a-4382-b05f-139114b4bfe7\") " Sep 09 21:02:16.714305 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:16.714202 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4079c714-876a-4382-b05f-139114b4bfe7-extensions-socket-volume\") pod \"4079c714-876a-4382-b05f-139114b4bfe7\" (UID: \"4079c714-876a-4382-b05f-139114b4bfe7\") " Sep 09 21:02:16.714590 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:16.714557 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4079c714-876a-4382-b05f-139114b4bfe7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4079c714-876a-4382-b05f-139114b4bfe7" (UID: "4079c714-876a-4382-b05f-139114b4bfe7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:02:16.716093 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:16.716075 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4079c714-876a-4382-b05f-139114b4bfe7-kube-api-access-hlzvn" (OuterVolumeSpecName: "kube-api-access-hlzvn") pod "4079c714-876a-4382-b05f-139114b4bfe7" (UID: "4079c714-876a-4382-b05f-139114b4bfe7"). InnerVolumeSpecName "kube-api-access-hlzvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:02:16.815230 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:16.815187 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlzvn\" (UniqueName: \"kubernetes.io/projected/4079c714-876a-4382-b05f-139114b4bfe7-kube-api-access-hlzvn\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 21:02:16.815230 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:16.815224 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4079c714-876a-4382-b05f-139114b4bfe7-extensions-socket-volume\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 21:02:17.372692 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:17.372656 2562 generic.go:356] "Generic (PLEG): container finished" podID="4079c714-876a-4382-b05f-139114b4bfe7" containerID="32f8f31629b5bda968ebb752b11dfcfee8ab912220791428933673924b94af75" exitCode=0 Sep 09 21:02:17.373121 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:17.372705 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" Sep 09 21:02:17.373121 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:17.372744 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" event={"ID":"4079c714-876a-4382-b05f-139114b4bfe7","Type":"ContainerDied","Data":"32f8f31629b5bda968ebb752b11dfcfee8ab912220791428933673924b94af75"} Sep 09 21:02:17.373121 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:17.372800 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6" event={"ID":"4079c714-876a-4382-b05f-139114b4bfe7","Type":"ContainerDied","Data":"451083daa63d71fa862b2fd29d8df89813c85e59e4e4ab89d148cc67099983e3"} Sep 09 21:02:17.373121 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:17.372822 2562 scope.go:117] "RemoveContainer" containerID="32f8f31629b5bda968ebb752b11dfcfee8ab912220791428933673924b94af75" Sep 09 21:02:17.383868 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:17.382835 2562 scope.go:117] "RemoveContainer" containerID="32f8f31629b5bda968ebb752b11dfcfee8ab912220791428933673924b94af75" Sep 09 21:02:17.383961 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:02:17.383942 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32f8f31629b5bda968ebb752b11dfcfee8ab912220791428933673924b94af75\": container with ID starting with 32f8f31629b5bda968ebb752b11dfcfee8ab912220791428933673924b94af75 not found: ID does not exist" containerID="32f8f31629b5bda968ebb752b11dfcfee8ab912220791428933673924b94af75" Sep 09 21:02:17.384004 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:17.383969 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32f8f31629b5bda968ebb752b11dfcfee8ab912220791428933673924b94af75"} err="failed to get container status \"32f8f31629b5bda968ebb752b11dfcfee8ab912220791428933673924b94af75\": rpc error: code = NotFound desc = could not find container \"32f8f31629b5bda968ebb752b11dfcfee8ab912220791428933673924b94af75\": container with ID starting with 32f8f31629b5bda968ebb752b11dfcfee8ab912220791428933673924b94af75 not found: ID does not exist" Sep 09 21:02:17.399630 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:17.399606 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6"] Sep 09 21:02:17.403251 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:17.403209 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d4ms6"] Sep 09 21:02:17.639144 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:02:17.639067 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4079c714-876a-4382-b05f-139114b4bfe7" path="/var/lib/kubelet/pods/4079c714-876a-4382-b05f-139114b4bfe7/volumes" Sep 09 21:03:19.175491 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.175456 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65cf998749-mrclz"] Sep 09 21:03:19.175934 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.175818 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4079c714-876a-4382-b05f-139114b4bfe7" containerName="manager" Sep 09 21:03:19.175934 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.175828 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4079c714-876a-4382-b05f-139114b4bfe7" containerName="manager" Sep 09 21:03:19.175934 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.175895 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4079c714-876a-4382-b05f-139114b4bfe7" containerName="manager" Sep 09 21:03:19.178950 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.178933 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:19.181630 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.181603 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 09 21:03:19.182406 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.182385 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 09 21:03:19.182525 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.182426 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qkvvl\"" Sep 09 21:03:19.182525 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.182430 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 09 21:03:19.192155 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.192132 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65cf998749-mrclz"] Sep 09 21:03:19.226901 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.226865 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97snk\" (UniqueName: \"kubernetes.io/projected/5f57f90c-0232-4475-91c9-2ef6374fc2a8-kube-api-access-97snk\") pod \"maas-controller-65cf998749-mrclz\" (UID: \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\") " pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:19.227077 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.226908 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-metrics-certs\") pod \"maas-controller-65cf998749-mrclz\" (UID: \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\") " pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:19.227077 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.226973 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-webhook-cert\") pod \"maas-controller-65cf998749-mrclz\" (UID: \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\") " pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:19.328480 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.328419 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-webhook-cert\") pod \"maas-controller-65cf998749-mrclz\" (UID: \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\") " pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:19.328671 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.328519 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97snk\" (UniqueName: \"kubernetes.io/projected/5f57f90c-0232-4475-91c9-2ef6374fc2a8-kube-api-access-97snk\") pod \"maas-controller-65cf998749-mrclz\" (UID: \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\") " pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:19.328671 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.328547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-metrics-certs\") pod \"maas-controller-65cf998749-mrclz\" (UID: \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\") " pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:19.328671 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:03:19.328578 2562 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 09 21:03:19.328671 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:03:19.328640 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-webhook-cert podName:5f57f90c-0232-4475-91c9-2ef6374fc2a8 nodeName:}" failed. No retries permitted until 2026-09-09 21:03:19.828625438 +0000 UTC m=+776.667824546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-webhook-cert") pod "maas-controller-65cf998749-mrclz" (UID: "5f57f90c-0232-4475-91c9-2ef6374fc2a8") : secret "maas-controller-webhook-cert" not found Sep 09 21:03:19.328826 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:03:19.328678 2562 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 09 21:03:19.328826 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:03:19.328745 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-metrics-certs podName:5f57f90c-0232-4475-91c9-2ef6374fc2a8 nodeName:}" failed. No retries permitted until 2026-09-09 21:03:19.828726265 +0000 UTC m=+776.667925374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-metrics-certs") pod "maas-controller-65cf998749-mrclz" (UID: "5f57f90c-0232-4475-91c9-2ef6374fc2a8") : secret "maas-controller-metrics-tls" not found Sep 09 21:03:19.352058 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.352021 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97snk\" (UniqueName: \"kubernetes.io/projected/5f57f90c-0232-4475-91c9-2ef6374fc2a8-kube-api-access-97snk\") pod \"maas-controller-65cf998749-mrclz\" (UID: \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\") " pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:19.833045 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.833015 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-metrics-certs\") pod \"maas-controller-65cf998749-mrclz\" (UID: \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\") " pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:19.833227 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.833070 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-webhook-cert\") pod \"maas-controller-65cf998749-mrclz\" (UID: \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\") " pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:19.835669 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.835637 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-webhook-cert\") pod \"maas-controller-65cf998749-mrclz\" (UID: \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\") " pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:19.835812 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:19.835682 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-metrics-certs\") pod \"maas-controller-65cf998749-mrclz\" (UID: \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\") " pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:20.089773 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:20.089676 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:20.361651 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:20.361625 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65cf998749-mrclz"] Sep 09 21:03:20.363335 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:03:20.363309 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f57f90c_0232_4475_91c9_2ef6374fc2a8.slice/crio-ce426839e7d8883740ac29614e668454b92eafb3d36b3873ec6b5e48f1321e70 WatchSource:0}: Error finding container ce426839e7d8883740ac29614e668454b92eafb3d36b3873ec6b5e48f1321e70: Status 404 returned error can't find the container with id ce426839e7d8883740ac29614e668454b92eafb3d36b3873ec6b5e48f1321e70 Sep 09 21:03:20.674497 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:20.674411 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65cf998749-mrclz" event={"ID":"5f57f90c-0232-4475-91c9-2ef6374fc2a8","Type":"ContainerStarted","Data":"ce426839e7d8883740ac29614e668454b92eafb3d36b3873ec6b5e48f1321e70"} Sep 09 21:03:23.692009 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:23.691972 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65cf998749-mrclz" event={"ID":"5f57f90c-0232-4475-91c9-2ef6374fc2a8","Type":"ContainerStarted","Data":"d53a00717b2f9d7bf759601a65854130501a49adeb203f0e93ceb5a4411874e5"} Sep 09 21:03:23.692524 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:23.692087 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:23.731169 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:23.731115 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65cf998749-mrclz" podStartSLOduration=2.14717789 podStartE2EDuration="4.731096911s" totalImagesPullingTime="2.583919021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:03:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:23.730003836 +0000 UTC m=+780.569202982" watchObservedRunningTime="2026-09-09 21:03:23.731096911 +0000 UTC m=+780.570296089" Sep 09 21:03:34.699364 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:34.699333 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:34.852528 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:34.852492 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65cf998749-mrclz"] Sep 09 21:03:34.852742 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:34.852719 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65cf998749-mrclz" podUID="5f57f90c-0232-4475-91c9-2ef6374fc2a8" containerName="manager" containerID="cri-o://d53a00717b2f9d7bf759601a65854130501a49adeb203f0e93ceb5a4411874e5" gracePeriod=30 Sep 09 21:03:35.106697 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.106670 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:35.265405 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.265325 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-webhook-cert\") pod \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\" (UID: \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\") " Sep 09 21:03:35.265405 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.265381 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97snk\" (UniqueName: \"kubernetes.io/projected/5f57f90c-0232-4475-91c9-2ef6374fc2a8-kube-api-access-97snk\") pod \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\" (UID: \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\") " Sep 09 21:03:35.265610 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.265472 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-metrics-certs\") pod \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\" (UID: \"5f57f90c-0232-4475-91c9-2ef6374fc2a8\") " Sep 09 21:03:35.267790 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.267755 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5f57f90c-0232-4475-91c9-2ef6374fc2a8" (UID: "5f57f90c-0232-4475-91c9-2ef6374fc2a8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:03:35.267790 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.267774 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5f57f90c-0232-4475-91c9-2ef6374fc2a8" (UID: "5f57f90c-0232-4475-91c9-2ef6374fc2a8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:03:35.267790 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.267774 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f57f90c-0232-4475-91c9-2ef6374fc2a8-kube-api-access-97snk" (OuterVolumeSpecName: "kube-api-access-97snk") pod "5f57f90c-0232-4475-91c9-2ef6374fc2a8" (UID: "5f57f90c-0232-4475-91c9-2ef6374fc2a8"). InnerVolumeSpecName "kube-api-access-97snk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:03:35.366447 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.366414 2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-metrics-certs\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 21:03:35.366447 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.366443 2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f57f90c-0232-4475-91c9-2ef6374fc2a8-webhook-cert\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 21:03:35.366447 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.366452 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97snk\" (UniqueName: \"kubernetes.io/projected/5f57f90c-0232-4475-91c9-2ef6374fc2a8-kube-api-access-97snk\") on node \"ip-10-0-139-83.ec2.internal\" DevicePath \"\"" Sep 09 21:03:35.748342 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.748306 2562 generic.go:356] "Generic (PLEG): container finished" podID="5f57f90c-0232-4475-91c9-2ef6374fc2a8" containerID="d53a00717b2f9d7bf759601a65854130501a49adeb203f0e93ceb5a4411874e5" exitCode=0 Sep 09 21:03:35.748723 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.748354 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65cf998749-mrclz" Sep 09 21:03:35.748723 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.748395 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65cf998749-mrclz" event={"ID":"5f57f90c-0232-4475-91c9-2ef6374fc2a8","Type":"ContainerDied","Data":"d53a00717b2f9d7bf759601a65854130501a49adeb203f0e93ceb5a4411874e5"} Sep 09 21:03:35.748723 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.748451 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65cf998749-mrclz" event={"ID":"5f57f90c-0232-4475-91c9-2ef6374fc2a8","Type":"ContainerDied","Data":"ce426839e7d8883740ac29614e668454b92eafb3d36b3873ec6b5e48f1321e70"} Sep 09 21:03:35.748723 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.748474 2562 scope.go:117] "RemoveContainer" containerID="d53a00717b2f9d7bf759601a65854130501a49adeb203f0e93ceb5a4411874e5" Sep 09 21:03:35.757107 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.757093 2562 scope.go:117] "RemoveContainer" containerID="d53a00717b2f9d7bf759601a65854130501a49adeb203f0e93ceb5a4411874e5" Sep 09 21:03:35.757377 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:03:35.757356 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d53a00717b2f9d7bf759601a65854130501a49adeb203f0e93ceb5a4411874e5\": container with ID starting with d53a00717b2f9d7bf759601a65854130501a49adeb203f0e93ceb5a4411874e5 not found: ID does not exist" containerID="d53a00717b2f9d7bf759601a65854130501a49adeb203f0e93ceb5a4411874e5" Sep 09 21:03:35.757438 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.757386 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d53a00717b2f9d7bf759601a65854130501a49adeb203f0e93ceb5a4411874e5"} err="failed to get container status \"d53a00717b2f9d7bf759601a65854130501a49adeb203f0e93ceb5a4411874e5\": rpc error: code = NotFound desc = could not find container \"d53a00717b2f9d7bf759601a65854130501a49adeb203f0e93ceb5a4411874e5\": container with ID starting with d53a00717b2f9d7bf759601a65854130501a49adeb203f0e93ceb5a4411874e5 not found: ID does not exist" Sep 09 21:03:35.766772 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.766750 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65cf998749-mrclz"] Sep 09 21:03:35.770500 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:35.770480 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65cf998749-mrclz"] Sep 09 21:03:37.643718 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:37.643681 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f57f90c-0232-4475-91c9-2ef6374fc2a8" path="/var/lib/kubelet/pods/5f57f90c-0232-4475-91c9-2ef6374fc2a8/volumes" Sep 09 21:03:56.913502 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:56.913466 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hcgf8"] Sep 09 21:03:56.914048 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:56.914031 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f57f90c-0232-4475-91c9-2ef6374fc2a8" containerName="manager" Sep 09 21:03:56.914107 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:56.914050 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f57f90c-0232-4475-91c9-2ef6374fc2a8" containerName="manager" Sep 09 21:03:56.914149 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:56.914108 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f57f90c-0232-4475-91c9-2ef6374fc2a8" containerName="manager" Sep 09 21:03:56.917331 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:56.917310 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-2pfrd"] Sep 09 21:03:56.917460 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:56.917443 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hcgf8" Sep 09 21:03:56.919759 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:56.919740 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 09 21:03:56.919854 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:56.919812 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-qpt9q\"" Sep 09 21:03:56.920578 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:56.920562 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-2pfrd" Sep 09 21:03:56.927892 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:56.927860 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hcgf8"] Sep 09 21:03:56.930069 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:56.930050 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-2pfrd"] Sep 09 21:03:56.940516 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:56.940490 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2vl2\" (UniqueName: \"kubernetes.io/projected/142967f1-89cf-48be-877a-22461c043ada-kube-api-access-h2vl2\") pod \"payload-processing-5b54b7fbb7-2pfrd\" (UID: \"142967f1-89cf-48be-877a-22461c043ada\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-2pfrd" Sep 09 21:03:56.940653 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:56.940571 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wvsh\" (UniqueName: \"kubernetes.io/projected/ac750ad1-5a58-43cb-b19a-53bf7c4ed6af-kube-api-access-6wvsh\") pod \"payload-pre-processing-59fbc47bf8-hcgf8\" (UID: \"ac750ad1-5a58-43cb-b19a-53bf7c4ed6af\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hcgf8" Sep 09 21:03:56.940723 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:56.940685 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac750ad1-5a58-43cb-b19a-53bf7c4ed6af-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hcgf8\" (UID: \"ac750ad1-5a58-43cb-b19a-53bf7c4ed6af\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hcgf8" Sep 09 21:03:56.940778 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:56.940726 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/142967f1-89cf-48be-877a-22461c043ada-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-2pfrd\" (UID: \"142967f1-89cf-48be-877a-22461c043ada\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-2pfrd" Sep 09 21:03:57.041243 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:57.041213 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2vl2\" (UniqueName: \"kubernetes.io/projected/142967f1-89cf-48be-877a-22461c043ada-kube-api-access-h2vl2\") pod \"payload-processing-5b54b7fbb7-2pfrd\" (UID: \"142967f1-89cf-48be-877a-22461c043ada\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-2pfrd" Sep 09 21:03:57.041429 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:57.041270 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wvsh\" (UniqueName: \"kubernetes.io/projected/ac750ad1-5a58-43cb-b19a-53bf7c4ed6af-kube-api-access-6wvsh\") pod \"payload-pre-processing-59fbc47bf8-hcgf8\" (UID: \"ac750ad1-5a58-43cb-b19a-53bf7c4ed6af\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hcgf8" Sep 09 21:03:57.041429 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:57.041317 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac750ad1-5a58-43cb-b19a-53bf7c4ed6af-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hcgf8\" (UID: \"ac750ad1-5a58-43cb-b19a-53bf7c4ed6af\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hcgf8" Sep 09 21:03:57.041429 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:57.041342 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/142967f1-89cf-48be-877a-22461c043ada-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-2pfrd\" (UID: \"142967f1-89cf-48be-877a-22461c043ada\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-2pfrd" Sep 09 21:03:57.042095 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:57.042025 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac750ad1-5a58-43cb-b19a-53bf7c4ed6af-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hcgf8\" (UID: \"ac750ad1-5a58-43cb-b19a-53bf7c4ed6af\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hcgf8" Sep 09 21:03:57.042095 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:57.042033 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/142967f1-89cf-48be-877a-22461c043ada-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-2pfrd\" (UID: \"142967f1-89cf-48be-877a-22461c043ada\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-2pfrd" Sep 09 21:03:57.052455 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:57.052425 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2vl2\" (UniqueName: \"kubernetes.io/projected/142967f1-89cf-48be-877a-22461c043ada-kube-api-access-h2vl2\") pod \"payload-processing-5b54b7fbb7-2pfrd\" (UID: \"142967f1-89cf-48be-877a-22461c043ada\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-2pfrd" Sep 09 21:03:57.055107 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:57.055078 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wvsh\" (UniqueName: \"kubernetes.io/projected/ac750ad1-5a58-43cb-b19a-53bf7c4ed6af-kube-api-access-6wvsh\") pod \"payload-pre-processing-59fbc47bf8-hcgf8\" (UID: \"ac750ad1-5a58-43cb-b19a-53bf7c4ed6af\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hcgf8" Sep 09 21:03:57.230700 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:57.230664 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hcgf8" Sep 09 21:03:57.237751 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:57.237733 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-2pfrd" Sep 09 21:03:57.513167 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:57.513142 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hcgf8"] Sep 09 21:03:57.515841 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:03:57.515814 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac750ad1_5a58_43cb_b19a_53bf7c4ed6af.slice/crio-e96ceff26483a276d82868bbcb6c00ca6b43ad68c48b8bdfcc73375b75faf010 WatchSource:0}: Error finding container e96ceff26483a276d82868bbcb6c00ca6b43ad68c48b8bdfcc73375b75faf010: Status 404 returned error can't find the container with id e96ceff26483a276d82868bbcb6c00ca6b43ad68c48b8bdfcc73375b75faf010 Sep 09 21:03:57.528123 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:57.528103 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-2pfrd"] Sep 09 21:03:57.529360 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:03:57.529334 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod142967f1_89cf_48be_877a_22461c043ada.slice/crio-a572be863b5c633090f8260ad29e7faf81199edcf6465c66a0a3361377fbe6d6 WatchSource:0}: Error finding container a572be863b5c633090f8260ad29e7faf81199edcf6465c66a0a3361377fbe6d6: Status 404 returned error can't find the container with id a572be863b5c633090f8260ad29e7faf81199edcf6465c66a0a3361377fbe6d6 Sep 09 21:03:57.851570 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:57.851481 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-2pfrd" event={"ID":"142967f1-89cf-48be-877a-22461c043ada","Type":"ContainerStarted","Data":"a572be863b5c633090f8260ad29e7faf81199edcf6465c66a0a3361377fbe6d6"} Sep 09 21:03:57.853461 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:57.853418 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hcgf8" event={"ID":"ac750ad1-5a58-43cb-b19a-53bf7c4ed6af","Type":"ContainerStarted","Data":"e96ceff26483a276d82868bbcb6c00ca6b43ad68c48b8bdfcc73375b75faf010"} Sep 09 21:03:59.870668 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:59.870630 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-2pfrd" event={"ID":"142967f1-89cf-48be-877a-22461c043ada","Type":"ContainerStarted","Data":"79fe974a98a0b9e9c9fce5b8004f77033004602239a47e0f5f7d1a4f6a07a18b"} Sep 09 21:03:59.871154 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:59.870702 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-2pfrd" Sep 09 21:03:59.872220 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:59.872197 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hcgf8" event={"ID":"ac750ad1-5a58-43cb-b19a-53bf7c4ed6af","Type":"ContainerStarted","Data":"588f7b1fc5921c6e7474af2aee1a03f796d03093d2b49421bd91538263ca102c"} Sep 09 21:03:59.872346 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:59.872297 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hcgf8" Sep 09 21:03:59.887373 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:59.887328 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-2pfrd" podStartSLOduration=2.001582787 podStartE2EDuration="3.887314535s" totalImagesPullingTime="1.885731748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:03:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:59.885946496 +0000 UTC m=+816.725145638" watchObservedRunningTime="2026-09-09 21:03:59.887314535 +0000 UTC m=+816.726513664" Sep 09 21:03:59.903688 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:03:59.903644 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hcgf8" podStartSLOduration=2.010333835 podStartE2EDuration="3.903632866s" totalImagesPullingTime="1.893299031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:03:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:59.902638806 +0000 UTC m=+816.741837953" watchObservedRunningTime="2026-09-09 21:03:59.903632866 +0000 UTC m=+816.742831995" Sep 09 21:04:10.879474 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:10.879448 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-2pfrd" Sep 09 21:04:10.879927 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:10.879517 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hcgf8" Sep 09 21:04:22.976152 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:22.976116 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs"] Sep 09 21:04:22.980101 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:22.980077 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:22.982668 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:22.982644 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 09 21:04:22.983590 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:22.983571 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5ztcr\"" Sep 09 21:04:22.983590 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:22.983575 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 09 21:04:22.983792 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:22.983608 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 09 21:04:22.992872 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:22.992845 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs"] Sep 09 21:04:23.053913 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.053876 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q85h\" (UniqueName: \"kubernetes.io/projected/cc77e860-7a77-4800-83d3-42083f1a952b-kube-api-access-7q85h\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.054091 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.053990 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc77e860-7a77-4800-83d3-42083f1a952b-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.054091 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.054061 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc77e860-7a77-4800-83d3-42083f1a952b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.054179 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.054124 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc77e860-7a77-4800-83d3-42083f1a952b-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.054179 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.054160 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc77e860-7a77-4800-83d3-42083f1a952b-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.054179 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.054175 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc77e860-7a77-4800-83d3-42083f1a952b-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.155656 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.155621 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc77e860-7a77-4800-83d3-42083f1a952b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.155815 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.155665 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc77e860-7a77-4800-83d3-42083f1a952b-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.155815 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.155687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc77e860-7a77-4800-83d3-42083f1a952b-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.155815 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.155703 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc77e860-7a77-4800-83d3-42083f1a952b-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.155815 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.155744 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q85h\" (UniqueName: \"kubernetes.io/projected/cc77e860-7a77-4800-83d3-42083f1a952b-kube-api-access-7q85h\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.155815 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.155761 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc77e860-7a77-4800-83d3-42083f1a952b-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.156125 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.156098 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc77e860-7a77-4800-83d3-42083f1a952b-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.156268 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.156097 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc77e860-7a77-4800-83d3-42083f1a952b-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.156268 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.156177 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc77e860-7a77-4800-83d3-42083f1a952b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.158293 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.158268 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc77e860-7a77-4800-83d3-42083f1a952b-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.158439 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.158420 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc77e860-7a77-4800-83d3-42083f1a952b-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.163925 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.163905 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q85h\" (UniqueName: \"kubernetes.io/projected/cc77e860-7a77-4800-83d3-42083f1a952b-kube-api-access-7q85h\") pod \"e2e-distinct-simulated-kserve-6685d86975-7w6gs\" (UID: \"cc77e860-7a77-4800-83d3-42083f1a952b\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.292147 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.292066 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:23.569351 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.569323 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs"] Sep 09 21:04:23.571083 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:04:23.571056 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc77e860_7a77_4800_83d3_42083f1a952b.slice/crio-6c14d11709bd3f828df19d711e6ed7e7716cbeb807f5b85283710ec20af4a1e5 WatchSource:0}: Error finding container 6c14d11709bd3f828df19d711e6ed7e7716cbeb807f5b85283710ec20af4a1e5: Status 404 returned error can't find the container with id 6c14d11709bd3f828df19d711e6ed7e7716cbeb807f5b85283710ec20af4a1e5 Sep 09 21:04:23.991276 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:23.991207 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" event={"ID":"cc77e860-7a77-4800-83d3-42083f1a952b","Type":"ContainerStarted","Data":"6c14d11709bd3f828df19d711e6ed7e7716cbeb807f5b85283710ec20af4a1e5"} Sep 09 21:04:24.997819 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:24.997788 2562 generic.go:356] "Generic (PLEG): container finished" podID="cc77e860-7a77-4800-83d3-42083f1a952b" containerID="debea55a20db4e7d757afc3ac7ba5c709f867575a06a21bb0071df7054017de2" exitCode=0 Sep 09 21:04:24.998207 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:24.997881 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" event={"ID":"cc77e860-7a77-4800-83d3-42083f1a952b","Type":"ContainerDied","Data":"debea55a20db4e7d757afc3ac7ba5c709f867575a06a21bb0071df7054017de2"} Sep 09 21:04:24.998207 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:24.998087 2562 scope.go:117] "RemoveContainer" containerID="debea55a20db4e7d757afc3ac7ba5c709f867575a06a21bb0071df7054017de2" Sep 09 21:04:26.004818 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:26.004769 2562 generic.go:356] "Generic (PLEG): container finished" podID="cc77e860-7a77-4800-83d3-42083f1a952b" containerID="a8f662186ff6b095e11bc85c25a3e34280174e278888aff4cfd9600fd9a04032" exitCode=0 Sep 09 21:04:26.005333 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:26.004855 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" event={"ID":"cc77e860-7a77-4800-83d3-42083f1a952b","Type":"ContainerDied","Data":"a8f662186ff6b095e11bc85c25a3e34280174e278888aff4cfd9600fd9a04032"} Sep 09 21:04:26.005333 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:26.004913 2562 scope.go:117] "RemoveContainer" containerID="debea55a20db4e7d757afc3ac7ba5c709f867575a06a21bb0071df7054017de2" Sep 09 21:04:26.005333 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:26.005040 2562 scope.go:117] "RemoveContainer" containerID="a8f662186ff6b095e11bc85c25a3e34280174e278888aff4cfd9600fd9a04032" Sep 09 21:04:26.005333 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:04:26.005262 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:04:27.011620 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:27.011592 2562 scope.go:117] "RemoveContainer" containerID="a8f662186ff6b095e11bc85c25a3e34280174e278888aff4cfd9600fd9a04032" Sep 09 21:04:27.011986 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:04:27.011749 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:04:33.293190 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:33.293152 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:33.293614 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:33.293220 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:33.293614 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:33.293262 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:33.293614 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:33.293282 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:33.293717 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:33.293657 2562 scope.go:117] "RemoveContainer" containerID="a8f662186ff6b095e11bc85c25a3e34280174e278888aff4cfd9600fd9a04032" Sep 09 21:04:33.293857 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:04:33.293838 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:04:42.375080 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.375041 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h"] Sep 09 21:04:42.379964 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.379942 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.382405 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.382384 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 09 21:04:42.388063 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.388042 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h"] Sep 09 21:04:42.523006 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.522970 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.523169 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.523032 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.523169 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.523075 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.523169 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.523158 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.523322 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.523182 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.523322 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.523197 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jhg9\" (UniqueName: \"kubernetes.io/projected/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-kube-api-access-9jhg9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.623858 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.623828 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.624025 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.623884 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.624025 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.624009 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.624156 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.624043 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jhg9\" (UniqueName: \"kubernetes.io/projected/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-kube-api-access-9jhg9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.624156 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.624079 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.624316 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.624293 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.624392 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.624377 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.624452 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.624401 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.624518 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.624501 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.626229 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.626180 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.626680 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.626662 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.633912 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.633889 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jhg9\" (UniqueName: \"kubernetes.io/projected/7365818c-5ff9-4d94-8a45-f13a6d01e4c2-kube-api-access-9jhg9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zp86h\" (UID: \"7365818c-5ff9-4d94-8a45-f13a6d01e4c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.690865 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.690826 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:42.963094 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:42.962883 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h"] Sep 09 21:04:42.965466 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:04:42.965441 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7365818c_5ff9_4d94_8a45_f13a6d01e4c2.slice/crio-5d3557da0208e9640cdb3cd55fc34afb9ad344fb9549fe9a38af142601dfa679 WatchSource:0}: Error finding container 5d3557da0208e9640cdb3cd55fc34afb9ad344fb9549fe9a38af142601dfa679: Status 404 returned error can't find the container with id 5d3557da0208e9640cdb3cd55fc34afb9ad344fb9549fe9a38af142601dfa679 Sep 09 21:04:43.088101 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:43.088065 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" event={"ID":"7365818c-5ff9-4d94-8a45-f13a6d01e4c2","Type":"ContainerStarted","Data":"5d3557da0208e9640cdb3cd55fc34afb9ad344fb9549fe9a38af142601dfa679"} Sep 09 21:04:44.095587 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:44.095550 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" event={"ID":"7365818c-5ff9-4d94-8a45-f13a6d01e4c2","Type":"ContainerStarted","Data":"1c431128ddea8a55d72077cc5e3c55f479018008aabf0ba33f182df048b289a3"} Sep 09 21:04:44.120697 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:44.120653 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" podStartSLOduration=1.917739922 podStartE2EDuration="2.120637816s" totalImagesPullingTime="202.897894ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:04:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:04:44.118963979 +0000 UTC m=+860.958163110" watchObservedRunningTime="2026-09-09 21:04:44.120637816 +0000 UTC m=+860.959836946" Sep 09 21:04:47.633298 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:47.633268 2562 scope.go:117] "RemoveContainer" containerID="a8f662186ff6b095e11bc85c25a3e34280174e278888aff4cfd9600fd9a04032" Sep 09 21:04:48.117252 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:48.117205 2562 generic.go:356] "Generic (PLEG): container finished" podID="cc77e860-7a77-4800-83d3-42083f1a952b" containerID="208991aa2263daaf949a7dc02e7f5b96a83dee51d64311ac3d381ae2f76de104" exitCode=0 Sep 09 21:04:48.117415 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:48.117273 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" event={"ID":"cc77e860-7a77-4800-83d3-42083f1a952b","Type":"ContainerDied","Data":"208991aa2263daaf949a7dc02e7f5b96a83dee51d64311ac3d381ae2f76de104"} Sep 09 21:04:48.117415 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:48.117324 2562 scope.go:117] "RemoveContainer" containerID="a8f662186ff6b095e11bc85c25a3e34280174e278888aff4cfd9600fd9a04032" Sep 09 21:04:48.117691 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:48.117671 2562 scope.go:117] "RemoveContainer" containerID="208991aa2263daaf949a7dc02e7f5b96a83dee51d64311ac3d381ae2f76de104" Sep 09 21:04:48.117915 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:04:48.117891 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:04:52.691715 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:52.691687 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:52.692166 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:52.691739 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:52.708167 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:52.708138 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:53.151888 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:53.151857 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zp86h" Sep 09 21:04:53.292941 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:53.292907 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:53.293083 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:53.292970 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:53.293083 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:53.292985 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:53.293083 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:53.292999 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:04:53.293369 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:53.293356 2562 scope.go:117] "RemoveContainer" containerID="208991aa2263daaf949a7dc02e7f5b96a83dee51d64311ac3d381ae2f76de104" Sep 09 21:04:53.293566 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:04:53.293549 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:04:58.618158 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.615414 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6"] Sep 09 21:04:58.623156 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.623110 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.627499 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.627483 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 09 21:04:58.639789 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.639765 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6"] Sep 09 21:04:58.647313 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.647289 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73633935-aa97-47bf-b046-3077a14e5392-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.647403 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.647349 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73633935-aa97-47bf-b046-3077a14e5392-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.647403 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.647380 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73633935-aa97-47bf-b046-3077a14e5392-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.647479 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.647438 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73633935-aa97-47bf-b046-3077a14e5392-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.647513 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.647479 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-847jg\" (UniqueName: \"kubernetes.io/projected/73633935-aa97-47bf-b046-3077a14e5392-kube-api-access-847jg\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.647558 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.647541 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73633935-aa97-47bf-b046-3077a14e5392-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.748504 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.748475 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73633935-aa97-47bf-b046-3077a14e5392-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.748633 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.748523 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73633935-aa97-47bf-b046-3077a14e5392-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.748633 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.748551 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73633935-aa97-47bf-b046-3077a14e5392-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.748633 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.748567 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73633935-aa97-47bf-b046-3077a14e5392-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.748633 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.748599 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73633935-aa97-47bf-b046-3077a14e5392-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.748633 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.748625 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-847jg\" (UniqueName: \"kubernetes.io/projected/73633935-aa97-47bf-b046-3077a14e5392-kube-api-access-847jg\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.748913 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.748889 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73633935-aa97-47bf-b046-3077a14e5392-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.748913 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.748905 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73633935-aa97-47bf-b046-3077a14e5392-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.749025 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.748976 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73633935-aa97-47bf-b046-3077a14e5392-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.750864 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.750835 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73633935-aa97-47bf-b046-3077a14e5392-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.751193 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.751176 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73633935-aa97-47bf-b046-3077a14e5392-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.759366 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.759342 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-847jg\" (UniqueName: \"kubernetes.io/projected/73633935-aa97-47bf-b046-3077a14e5392-kube-api-access-847jg\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6\" (UID: \"73633935-aa97-47bf-b046-3077a14e5392\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:58.933137 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:58.933047 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:04:59.211378 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:04:59.211353 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6"] Sep 09 21:04:59.213002 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:04:59.212968 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73633935_aa97_47bf_b046_3077a14e5392.slice/crio-3646272287fcb9c39ec4daefd6bdbfb8c9b8fb3cdb42944e3093d683061970f3 WatchSource:0}: Error finding container 3646272287fcb9c39ec4daefd6bdbfb8c9b8fb3cdb42944e3093d683061970f3: Status 404 returned error can't find the container with id 3646272287fcb9c39ec4daefd6bdbfb8c9b8fb3cdb42944e3093d683061970f3 Sep 09 21:05:00.174965 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:00.174931 2562 generic.go:356] "Generic (PLEG): container finished" podID="73633935-aa97-47bf-b046-3077a14e5392" containerID="7c233750ac49694f60627f29ec0eaa475d0c06e5cd5d6e14b7d560764d1ba526" exitCode=0 Sep 09 21:05:00.175362 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:00.175030 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" event={"ID":"73633935-aa97-47bf-b046-3077a14e5392","Type":"ContainerDied","Data":"7c233750ac49694f60627f29ec0eaa475d0c06e5cd5d6e14b7d560764d1ba526"} Sep 09 21:05:00.175362 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:00.175089 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" event={"ID":"73633935-aa97-47bf-b046-3077a14e5392","Type":"ContainerStarted","Data":"3646272287fcb9c39ec4daefd6bdbfb8c9b8fb3cdb42944e3093d683061970f3"} Sep 09 21:05:00.175362 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:00.175346 2562 scope.go:117] "RemoveContainer" containerID="7c233750ac49694f60627f29ec0eaa475d0c06e5cd5d6e14b7d560764d1ba526" Sep 09 21:05:01.181846 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:01.181810 2562 generic.go:356] "Generic (PLEG): container finished" podID="73633935-aa97-47bf-b046-3077a14e5392" containerID="a0631dd2d611cebc607501346467ce547f9e3b6c5632ec48bf5d2ee31fc627c4" exitCode=0 Sep 09 21:05:01.182264 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:01.181880 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" event={"ID":"73633935-aa97-47bf-b046-3077a14e5392","Type":"ContainerDied","Data":"a0631dd2d611cebc607501346467ce547f9e3b6c5632ec48bf5d2ee31fc627c4"} Sep 09 21:05:01.182264 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:01.181933 2562 scope.go:117] "RemoveContainer" containerID="7c233750ac49694f60627f29ec0eaa475d0c06e5cd5d6e14b7d560764d1ba526" Sep 09 21:05:01.182264 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:01.182207 2562 scope.go:117] "RemoveContainer" containerID="a0631dd2d611cebc607501346467ce547f9e3b6c5632ec48bf5d2ee31fc627c4" Sep 09 21:05:01.182456 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:05:01.182413 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:05:02.189192 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:02.189161 2562 scope.go:117] "RemoveContainer" containerID="a0631dd2d611cebc607501346467ce547f9e3b6c5632ec48bf5d2ee31fc627c4" Sep 09 21:05:02.189594 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:05:02.189393 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:05:04.633080 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:04.633053 2562 scope.go:117] "RemoveContainer" containerID="208991aa2263daaf949a7dc02e7f5b96a83dee51d64311ac3d381ae2f76de104" Sep 09 21:05:04.633537 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:05:04.633230 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:05:08.933373 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:08.933332 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:05:08.933753 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:08.933396 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:05:08.933753 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:08.933415 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:05:08.933753 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:08.933462 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:05:08.933856 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:08.933765 2562 scope.go:117] "RemoveContainer" containerID="a0631dd2d611cebc607501346467ce547f9e3b6c5632ec48bf5d2ee31fc627c4" Sep 09 21:05:08.933958 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:05:08.933939 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:05:09.225604 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:09.225577 2562 scope.go:117] "RemoveContainer" containerID="a0631dd2d611cebc607501346467ce547f9e3b6c5632ec48bf5d2ee31fc627c4" Sep 09 21:05:09.225775 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:05:09.225757 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:05:16.633293 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:16.633201 2562 scope.go:117] "RemoveContainer" containerID="208991aa2263daaf949a7dc02e7f5b96a83dee51d64311ac3d381ae2f76de104" Sep 09 21:05:17.264461 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:17.264422 2562 generic.go:356] "Generic (PLEG): container finished" podID="cc77e860-7a77-4800-83d3-42083f1a952b" containerID="58f319a32a89c64968b3dc19e10f10e6d18f95b7dc4b16e4831fa659d6039e56" exitCode=0 Sep 09 21:05:17.264632 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:17.264502 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" event={"ID":"cc77e860-7a77-4800-83d3-42083f1a952b","Type":"ContainerDied","Data":"58f319a32a89c64968b3dc19e10f10e6d18f95b7dc4b16e4831fa659d6039e56"} Sep 09 21:05:17.264632 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:17.264568 2562 scope.go:117] "RemoveContainer" containerID="208991aa2263daaf949a7dc02e7f5b96a83dee51d64311ac3d381ae2f76de104" Sep 09 21:05:17.264884 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:17.264862 2562 scope.go:117] "RemoveContainer" containerID="58f319a32a89c64968b3dc19e10f10e6d18f95b7dc4b16e4831fa659d6039e56" Sep 09 21:05:17.265075 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:05:17.265059 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:05:21.639560 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:21.639533 2562 scope.go:117] "RemoveContainer" containerID="a0631dd2d611cebc607501346467ce547f9e3b6c5632ec48bf5d2ee31fc627c4" Sep 09 21:05:22.293166 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:22.293132 2562 generic.go:356] "Generic (PLEG): container finished" podID="73633935-aa97-47bf-b046-3077a14e5392" containerID="f3fb5ba4ccd8c5e03b06891bd403c7adcd7167a6e4cbc253bd6d8b85fbc6f7b2" exitCode=0 Sep 09 21:05:22.293166 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:22.293169 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" event={"ID":"73633935-aa97-47bf-b046-3077a14e5392","Type":"ContainerDied","Data":"f3fb5ba4ccd8c5e03b06891bd403c7adcd7167a6e4cbc253bd6d8b85fbc6f7b2"} Sep 09 21:05:22.293407 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:22.293260 2562 scope.go:117] "RemoveContainer" containerID="a0631dd2d611cebc607501346467ce547f9e3b6c5632ec48bf5d2ee31fc627c4" Sep 09 21:05:22.293600 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:22.293582 2562 scope.go:117] "RemoveContainer" containerID="f3fb5ba4ccd8c5e03b06891bd403c7adcd7167a6e4cbc253bd6d8b85fbc6f7b2" Sep 09 21:05:22.293810 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:05:22.293791 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:05:23.293008 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:23.292972 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:05:23.293419 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:23.293064 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:05:23.293419 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:23.293089 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:05:23.293419 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:23.293108 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:05:23.293419 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:23.293417 2562 scope.go:117] "RemoveContainer" containerID="58f319a32a89c64968b3dc19e10f10e6d18f95b7dc4b16e4831fa659d6039e56" Sep 09 21:05:23.293627 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:05:23.293603 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:05:24.304651 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:24.304620 2562 scope.go:117] "RemoveContainer" containerID="58f319a32a89c64968b3dc19e10f10e6d18f95b7dc4b16e4831fa659d6039e56" Sep 09 21:05:24.305113 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:05:24.304855 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:05:28.934091 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:28.934048 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:05:28.936524 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:28.934120 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:05:28.936524 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:28.934137 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:05:28.936524 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:28.934150 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:05:28.936524 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:28.934469 2562 scope.go:117] "RemoveContainer" containerID="f3fb5ba4ccd8c5e03b06891bd403c7adcd7167a6e4cbc253bd6d8b85fbc6f7b2" Sep 09 21:05:28.936524 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:05:28.934637 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:05:35.633596 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:35.633565 2562 scope.go:117] "RemoveContainer" containerID="58f319a32a89c64968b3dc19e10f10e6d18f95b7dc4b16e4831fa659d6039e56" Sep 09 21:05:35.634061 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:05:35.633736 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:05:41.633179 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:41.633145 2562 scope.go:117] "RemoveContainer" containerID="f3fb5ba4ccd8c5e03b06891bd403c7adcd7167a6e4cbc253bd6d8b85fbc6f7b2" Sep 09 21:05:41.633591 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:05:41.633352 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:05:48.633258 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:48.633211 2562 scope.go:117] "RemoveContainer" containerID="58f319a32a89c64968b3dc19e10f10e6d18f95b7dc4b16e4831fa659d6039e56" Sep 09 21:05:48.633657 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:05:48.633404 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:05:53.635783 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:53.635754 2562 scope.go:117] "RemoveContainer" containerID="f3fb5ba4ccd8c5e03b06891bd403c7adcd7167a6e4cbc253bd6d8b85fbc6f7b2" Sep 09 21:05:53.636151 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:53.636057 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:05:54.450116 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:54.450080 2562 generic.go:356] "Generic (PLEG): container finished" podID="73633935-aa97-47bf-b046-3077a14e5392" containerID="ddc43f41cac74fb862408d44d91e7fad160a505cb852a29c4b1366112cf9e34d" exitCode=0 Sep 09 21:05:54.450303 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:54.450148 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" event={"ID":"73633935-aa97-47bf-b046-3077a14e5392","Type":"ContainerDied","Data":"ddc43f41cac74fb862408d44d91e7fad160a505cb852a29c4b1366112cf9e34d"} Sep 09 21:05:54.450303 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:54.450207 2562 scope.go:117] "RemoveContainer" containerID="f3fb5ba4ccd8c5e03b06891bd403c7adcd7167a6e4cbc253bd6d8b85fbc6f7b2" Sep 09 21:05:54.450574 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:54.450558 2562 scope.go:117] "RemoveContainer" containerID="ddc43f41cac74fb862408d44d91e7fad160a505cb852a29c4b1366112cf9e34d" Sep 09 21:05:54.450750 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:05:54.450731 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:05:58.933510 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:58.933476 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:05:58.933992 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:58.933540 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:05:58.933992 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:58.933573 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:05:58.933992 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:58.933593 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:05:58.934155 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:05:58.934039 2562 scope.go:117] "RemoveContainer" containerID="ddc43f41cac74fb862408d44d91e7fad160a505cb852a29c4b1366112cf9e34d" Sep 09 21:05:58.934324 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:05:58.934298 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:06:02.632834 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:02.632797 2562 scope.go:117] "RemoveContainer" containerID="58f319a32a89c64968b3dc19e10f10e6d18f95b7dc4b16e4831fa659d6039e56" Sep 09 21:06:03.497049 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:03.497014 2562 generic.go:356] "Generic (PLEG): container finished" podID="cc77e860-7a77-4800-83d3-42083f1a952b" containerID="f0450c1ded06f0e3494ce518e74431e6c91cad5fe8e982615d0eb706da28d211" exitCode=0 Sep 09 21:06:03.497226 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:03.497086 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" event={"ID":"cc77e860-7a77-4800-83d3-42083f1a952b","Type":"ContainerDied","Data":"f0450c1ded06f0e3494ce518e74431e6c91cad5fe8e982615d0eb706da28d211"} Sep 09 21:06:03.497226 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:03.497142 2562 scope.go:117] "RemoveContainer" containerID="58f319a32a89c64968b3dc19e10f10e6d18f95b7dc4b16e4831fa659d6039e56" Sep 09 21:06:03.497499 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:03.497474 2562 scope.go:117] "RemoveContainer" containerID="f0450c1ded06f0e3494ce518e74431e6c91cad5fe8e982615d0eb706da28d211" Sep 09 21:06:03.497698 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:06:03.497677 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:06:12.632949 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:12.632918 2562 scope.go:117] "RemoveContainer" containerID="ddc43f41cac74fb862408d44d91e7fad160a505cb852a29c4b1366112cf9e34d" Sep 09 21:06:12.633421 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:06:12.633108 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:06:13.292839 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:13.292804 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:06:13.293022 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:13.292881 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:06:13.293022 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:13.292906 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:06:13.293022 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:13.292925 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:06:13.293314 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:13.293296 2562 scope.go:117] "RemoveContainer" containerID="f0450c1ded06f0e3494ce518e74431e6c91cad5fe8e982615d0eb706da28d211" Sep 09 21:06:13.293501 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:06:13.293483 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:06:24.633214 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:24.633185 2562 scope.go:117] "RemoveContainer" containerID="f0450c1ded06f0e3494ce518e74431e6c91cad5fe8e982615d0eb706da28d211" Sep 09 21:06:24.633700 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:06:24.633428 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:06:26.633498 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:26.633470 2562 scope.go:117] "RemoveContainer" containerID="ddc43f41cac74fb862408d44d91e7fad160a505cb852a29c4b1366112cf9e34d" Sep 09 21:06:26.633866 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:06:26.633639 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:06:37.633818 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:37.633787 2562 scope.go:117] "RemoveContainer" containerID="f0450c1ded06f0e3494ce518e74431e6c91cad5fe8e982615d0eb706da28d211" Sep 09 21:06:37.634417 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:06:37.633963 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:06:40.633216 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:40.633188 2562 scope.go:117] "RemoveContainer" containerID="ddc43f41cac74fb862408d44d91e7fad160a505cb852a29c4b1366112cf9e34d" Sep 09 21:06:41.680187 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:41.680152 2562 generic.go:356] "Generic (PLEG): container finished" podID="73633935-aa97-47bf-b046-3077a14e5392" containerID="73aa529ada0960dc99b13d6f21bff9b664215bd6576d40d69dcc51c52d01711e" exitCode=0 Sep 09 21:06:41.680615 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:41.680221 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" event={"ID":"73633935-aa97-47bf-b046-3077a14e5392","Type":"ContainerDied","Data":"73aa529ada0960dc99b13d6f21bff9b664215bd6576d40d69dcc51c52d01711e"} Sep 09 21:06:41.680615 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:41.680296 2562 scope.go:117] "RemoveContainer" containerID="ddc43f41cac74fb862408d44d91e7fad160a505cb852a29c4b1366112cf9e34d" Sep 09 21:06:41.680740 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:41.680685 2562 scope.go:117] "RemoveContainer" containerID="73aa529ada0960dc99b13d6f21bff9b664215bd6576d40d69dcc51c52d01711e" Sep 09 21:06:41.680888 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:06:41.680870 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:06:48.933899 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:48.933809 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:06:48.933899 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:48.933883 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:06:48.934437 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:48.933907 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:06:48.934437 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:48.933926 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:06:48.934437 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:48.934313 2562 scope.go:117] "RemoveContainer" containerID="73aa529ada0960dc99b13d6f21bff9b664215bd6576d40d69dcc51c52d01711e" Sep 09 21:06:48.934542 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:06:48.934503 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:06:51.633761 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:06:51.633733 2562 scope.go:117] "RemoveContainer" containerID="f0450c1ded06f0e3494ce518e74431e6c91cad5fe8e982615d0eb706da28d211" Sep 09 21:06:51.634157 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:06:51.633922 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:07:02.633431 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:02.633396 2562 scope.go:117] "RemoveContainer" containerID="73aa529ada0960dc99b13d6f21bff9b664215bd6576d40d69dcc51c52d01711e" Sep 09 21:07:02.634003 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:07:02.633602 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:07:04.633634 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:04.633608 2562 scope.go:117] "RemoveContainer" containerID="f0450c1ded06f0e3494ce518e74431e6c91cad5fe8e982615d0eb706da28d211" Sep 09 21:07:04.634011 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:07:04.633797 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:07:15.633193 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:15.633163 2562 scope.go:117] "RemoveContainer" containerID="f0450c1ded06f0e3494ce518e74431e6c91cad5fe8e982615d0eb706da28d211" Sep 09 21:07:15.633659 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:07:15.633381 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:07:15.633659 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:15.633413 2562 scope.go:117] "RemoveContainer" containerID="73aa529ada0960dc99b13d6f21bff9b664215bd6576d40d69dcc51c52d01711e" Sep 09 21:07:15.633659 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:07:15.633602 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:07:28.633233 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:28.633203 2562 scope.go:117] "RemoveContainer" containerID="f0450c1ded06f0e3494ce518e74431e6c91cad5fe8e982615d0eb706da28d211" Sep 09 21:07:29.633692 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:29.633662 2562 scope.go:117] "RemoveContainer" containerID="73aa529ada0960dc99b13d6f21bff9b664215bd6576d40d69dcc51c52d01711e" Sep 09 21:07:29.634104 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:07:29.633830 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:07:29.915511 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:29.915430 2562 generic.go:356] "Generic (PLEG): container finished" podID="cc77e860-7a77-4800-83d3-42083f1a952b" containerID="1bda35652734bcf189bf1490061e825a58793938600b769940636892a501ab91" exitCode=0 Sep 09 21:07:29.915511 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:29.915498 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" event={"ID":"cc77e860-7a77-4800-83d3-42083f1a952b","Type":"ContainerDied","Data":"1bda35652734bcf189bf1490061e825a58793938600b769940636892a501ab91"} Sep 09 21:07:29.915702 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:29.915557 2562 scope.go:117] "RemoveContainer" containerID="f0450c1ded06f0e3494ce518e74431e6c91cad5fe8e982615d0eb706da28d211" Sep 09 21:07:29.915938 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:29.915921 2562 scope.go:117] "RemoveContainer" containerID="1bda35652734bcf189bf1490061e825a58793938600b769940636892a501ab91" Sep 09 21:07:29.916162 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:07:29.916138 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:07:33.292829 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:33.292798 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:07:33.293221 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:33.292853 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:07:33.293221 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:33.292877 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:07:33.293221 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:33.292893 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" Sep 09 21:07:33.293221 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:33.293213 2562 scope.go:117] "RemoveContainer" containerID="1bda35652734bcf189bf1490061e825a58793938600b769940636892a501ab91" Sep 09 21:07:33.293421 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:07:33.293402 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:07:41.633111 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:41.633079 2562 scope.go:117] "RemoveContainer" containerID="73aa529ada0960dc99b13d6f21bff9b664215bd6576d40d69dcc51c52d01711e" Sep 09 21:07:41.633581 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:07:41.633284 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:07:43.635196 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:43.635172 2562 scope.go:117] "RemoveContainer" containerID="1bda35652734bcf189bf1490061e825a58793938600b769940636892a501ab91" Sep 09 21:07:43.635600 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:07:43.635364 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:07:55.633609 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:55.633581 2562 scope.go:117] "RemoveContainer" containerID="73aa529ada0960dc99b13d6f21bff9b664215bd6576d40d69dcc51c52d01711e" Sep 09 21:07:55.634050 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:07:55.633736 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:07:58.633401 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:07:58.633370 2562 scope.go:117] "RemoveContainer" containerID="1bda35652734bcf189bf1490061e825a58793938600b769940636892a501ab91" Sep 09 21:07:58.633783 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:07:58.633518 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:08:07.632962 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:07.632926 2562 scope.go:117] "RemoveContainer" containerID="73aa529ada0960dc99b13d6f21bff9b664215bd6576d40d69dcc51c52d01711e" Sep 09 21:08:08.106500 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:08.106470 2562 generic.go:356] "Generic (PLEG): container finished" podID="73633935-aa97-47bf-b046-3077a14e5392" containerID="70871f21d63390578a02ab75ed31340149aafb9c6e22a5d6735145cf3c18d800" exitCode=0 Sep 09 21:08:08.106666 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:08.106548 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" event={"ID":"73633935-aa97-47bf-b046-3077a14e5392","Type":"ContainerDied","Data":"70871f21d63390578a02ab75ed31340149aafb9c6e22a5d6735145cf3c18d800"} Sep 09 21:08:08.106666 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:08.106636 2562 scope.go:117] "RemoveContainer" containerID="73aa529ada0960dc99b13d6f21bff9b664215bd6576d40d69dcc51c52d01711e" Sep 09 21:08:08.106988 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:08.106967 2562 scope.go:117] "RemoveContainer" containerID="70871f21d63390578a02ab75ed31340149aafb9c6e22a5d6735145cf3c18d800" Sep 09 21:08:08.107178 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:08:08.107158 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:08:08.933394 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:08.933362 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:08:08.933762 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:08.933416 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:08:08.933762 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:08.933431 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:08:08.933762 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:08.933445 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" Sep 09 21:08:09.113499 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:09.113475 2562 scope.go:117] "RemoveContainer" containerID="70871f21d63390578a02ab75ed31340149aafb9c6e22a5d6735145cf3c18d800" Sep 09 21:08:09.113672 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:08:09.113652 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:08:09.633355 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:09.633326 2562 scope.go:117] "RemoveContainer" containerID="1bda35652734bcf189bf1490061e825a58793938600b769940636892a501ab91" Sep 09 21:08:09.633523 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:08:09.633506 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:08:20.633806 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:20.633727 2562 scope.go:117] "RemoveContainer" containerID="1bda35652734bcf189bf1490061e825a58793938600b769940636892a501ab91" Sep 09 21:08:20.634297 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:08:20.633919 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:08:21.633598 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:21.633569 2562 scope.go:117] "RemoveContainer" containerID="70871f21d63390578a02ab75ed31340149aafb9c6e22a5d6735145cf3c18d800" Sep 09 21:08:21.633787 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:08:21.633764 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:08:31.633640 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:31.633612 2562 scope.go:117] "RemoveContainer" containerID="1bda35652734bcf189bf1490061e825a58793938600b769940636892a501ab91" Sep 09 21:08:31.634112 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:08:31.633768 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:08:33.635205 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:33.635178 2562 scope.go:117] "RemoveContainer" containerID="70871f21d63390578a02ab75ed31340149aafb9c6e22a5d6735145cf3c18d800" Sep 09 21:08:33.635606 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:08:33.635368 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:08:44.633465 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:44.633427 2562 scope.go:117] "RemoveContainer" containerID="70871f21d63390578a02ab75ed31340149aafb9c6e22a5d6735145cf3c18d800" Sep 09 21:08:44.634056 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:08:44.633681 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:08:46.633378 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:46.633342 2562 scope.go:117] "RemoveContainer" containerID="1bda35652734bcf189bf1490061e825a58793938600b769940636892a501ab91" Sep 09 21:08:46.633766 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:08:46.633525 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:08:55.632859 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:08:55.632821 2562 scope.go:117] "RemoveContainer" containerID="70871f21d63390578a02ab75ed31340149aafb9c6e22a5d6735145cf3c18d800" Sep 09 21:08:55.633470 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:08:55.633004 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:09:00.633410 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:00.633372 2562 scope.go:117] "RemoveContainer" containerID="1bda35652734bcf189bf1490061e825a58793938600b769940636892a501ab91" Sep 09 21:09:00.633866 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:09:00.633616 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:09:06.633573 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:06.633547 2562 scope.go:117] "RemoveContainer" containerID="70871f21d63390578a02ab75ed31340149aafb9c6e22a5d6735145cf3c18d800" Sep 09 21:09:06.634013 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:09:06.633726 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:09:10.349842 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:10.349802 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_73633935-aa97-47bf-b046-3077a14e5392/main/5.log" Sep 09 21:09:10.463034 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:10.463001 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_73633935-aa97-47bf-b046-3077a14e5392/main/5.log" Sep 09 21:09:13.637302 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:13.637277 2562 scope.go:117] "RemoveContainer" containerID="1bda35652734bcf189bf1490061e825a58793938600b769940636892a501ab91" Sep 09 21:09:13.637766 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:09:13.637449 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:09:17.032662 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:17.032626 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-8trts_3b65b188-5a8e-4307-9d22-4ffd70c6b7c6/server/0.log" Sep 09 21:09:17.160978 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:17.160946 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-ckv66_8d78d46f-4740-404e-984e-f647488edb65/manager/0.log" Sep 09 21:09:17.279383 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:17.279356 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cf9547dff-glg6x_0178f901-e8d9-42b6-b07a-662dadc9ace3/copy-manifests/0.log" Sep 09 21:09:17.289289 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:17.289200 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cf9547dff-glg6x_0178f901-e8d9-42b6-b07a-662dadc9ace3/manager/0.log" Sep 09 21:09:18.632836 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:18.632809 2562 scope.go:117] "RemoveContainer" containerID="70871f21d63390578a02ab75ed31340149aafb9c6e22a5d6735145cf3c18d800" Sep 09 21:09:18.633224 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:09:18.632984 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:09:19.169638 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:19.169614 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-lz82c_b9afc124-9af3-4e6f-9c70-d9f911d1c7de/manager/0.log" Sep 09 21:09:19.380222 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:19.380185 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-vp9dp_733a6c42-1ad7-4ca8-bb1c-44f80595149e/manager/0.log" Sep 09 21:09:19.714082 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:19.714052 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4_d64d0526-7ac1-4363-8fa9-b10cb0e9d334/istio-proxy/0.log" Sep 09 21:09:19.830366 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:19.830337 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-s7g9g_2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c/discovery/0.log" Sep 09 21:09:19.936933 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:19.936911 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55f8b7786b-7rk56_ba348cb1-cd13-4c92-854e-b059cd5612c6/kube-auth-proxy/0.log" Sep 09 21:09:20.264065 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:20.264020 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hcgf8_ac750ad1-5a58-43cb-b19a-53bf7c4ed6af/payload-pre-processing/0.log" Sep 09 21:09:20.387409 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:20.387377 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-2pfrd_142967f1-89cf-48be-877a-22461c043ada/payload-processing/0.log" Sep 09 21:09:20.949030 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:20.949000 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-6685d86975-7w6gs_cc77e860-7a77-4800-83d3-42083f1a952b/main/5.log" Sep 09 21:09:21.169908 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:21.169878 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-zp86h_7365818c-5ff9-4d94-8a45-f13a6d01e4c2/main/0.log" Sep 09 21:09:21.386022 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:21.385995 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_73633935-aa97-47bf-b046-3077a14e5392/main/5.log" Sep 09 21:09:28.633554 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:28.633525 2562 scope.go:117] "RemoveContainer" containerID="1bda35652734bcf189bf1490061e825a58793938600b769940636892a501ab91" Sep 09 21:09:28.633993 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:09:28.633696 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:09:29.464708 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:29.464680 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-x59k7_79c3cb22-0efb-42bc-8697-ad6431a2a874/global-pull-secret-syncer/0.log" Sep 09 21:09:29.569881 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:29.569848 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mgrlj_5e788c00-eca3-4b87-98e5-89bd93f0abd0/konnectivity-agent/0.log" Sep 09 21:09:29.636008 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:29.635976 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-83.ec2.internal_2df482739528dd8ed1304969d8f65a5c/haproxy/0.log" Sep 09 21:09:33.435578 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:33.435544 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-lz82c_b9afc124-9af3-4e6f-9c70-d9f911d1c7de/manager/0.log" Sep 09 21:09:33.501169 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:33.501135 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-vp9dp_733a6c42-1ad7-4ca8-bb1c-44f80595149e/manager/0.log" Sep 09 21:09:33.635444 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:33.635417 2562 scope.go:117] "RemoveContainer" containerID="70871f21d63390578a02ab75ed31340149aafb9c6e22a5d6735145cf3c18d800" Sep 09 21:09:33.635616 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:09:33.635597 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:09:35.344293 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:35.344257 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a847ee6-e620-4b7c-84c6-cc1290184610/alertmanager/0.log" Sep 09 21:09:35.371848 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:35.371813 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a847ee6-e620-4b7c-84c6-cc1290184610/config-reloader/0.log" Sep 09 21:09:35.394138 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:35.394111 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a847ee6-e620-4b7c-84c6-cc1290184610/kube-rbac-proxy-web/0.log" Sep 09 21:09:35.418211 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:35.418191 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a847ee6-e620-4b7c-84c6-cc1290184610/kube-rbac-proxy/0.log" Sep 09 21:09:35.445991 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:35.445967 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a847ee6-e620-4b7c-84c6-cc1290184610/kube-rbac-proxy-metric/0.log" Sep 09 21:09:35.470937 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:35.470915 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a847ee6-e620-4b7c-84c6-cc1290184610/prom-label-proxy/0.log" Sep 09 21:09:35.492817 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:35.492800 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4a847ee6-e620-4b7c-84c6-cc1290184610/init-config-reloader/0.log" Sep 09 21:09:35.573818 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:35.573792 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-bsnjz_d7e332e8-9425-4403-8a37-60af9a8058ec/kube-state-metrics/0.log" Sep 09 21:09:35.600939 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:35.600879 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-bsnjz_d7e332e8-9425-4403-8a37-60af9a8058ec/kube-rbac-proxy-main/0.log" Sep 09 21:09:35.630402 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:35.630376 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-bsnjz_d7e332e8-9425-4403-8a37-60af9a8058ec/kube-rbac-proxy-self/0.log" Sep 09 21:09:35.725527 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:35.725507 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-24fjb_0fca05bf-c7fd-48ca-b5da-53be00d48a19/node-exporter/0.log" Sep 09 21:09:35.751846 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:35.751815 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-24fjb_0fca05bf-c7fd-48ca-b5da-53be00d48a19/kube-rbac-proxy/0.log" Sep 09 21:09:35.774596 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:35.774574 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-24fjb_0fca05bf-c7fd-48ca-b5da-53be00d48a19/init-textfile/0.log" Sep 09 21:09:36.388070 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:36.388043 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d7c6bc989-qxn8g_5a6791c7-e21c-4e3f-986f-2251bb52e447/telemeter-client/0.log" Sep 09 21:09:36.415559 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:36.415535 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d7c6bc989-qxn8g_5a6791c7-e21c-4e3f-986f-2251bb52e447/reload/0.log" Sep 09 21:09:36.442599 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:36.442573 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d7c6bc989-qxn8g_5a6791c7-e21c-4e3f-986f-2251bb52e447/kube-rbac-proxy/0.log" Sep 09 21:09:39.476434 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.476397 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm"] Sep 09 21:09:39.480256 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.480219 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:39.482816 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.482790 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5djh2\"/\"kube-root-ca.crt\"" Sep 09 21:09:39.482816 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.482790 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5djh2\"/\"default-dockercfg-vknfd\"" Sep 09 21:09:39.482988 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.482791 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5djh2\"/\"openshift-service-ca.crt\"" Sep 09 21:09:39.494722 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.494699 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm"] Sep 09 21:09:39.540663 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.540625 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3f87e6ae-db0b-4ba1-a637-ed5617892aff-proc\") pod \"perf-node-gather-daemonset-zgbfm\" (UID: \"3f87e6ae-db0b-4ba1-a637-ed5617892aff\") " pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:39.540792 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.540669 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f87e6ae-db0b-4ba1-a637-ed5617892aff-lib-modules\") pod \"perf-node-gather-daemonset-zgbfm\" (UID: \"3f87e6ae-db0b-4ba1-a637-ed5617892aff\") " pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:39.540792 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.540745 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z74wr\" (UniqueName: \"kubernetes.io/projected/3f87e6ae-db0b-4ba1-a637-ed5617892aff-kube-api-access-z74wr\") pod \"perf-node-gather-daemonset-zgbfm\" (UID: \"3f87e6ae-db0b-4ba1-a637-ed5617892aff\") " pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:39.540882 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.540799 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f87e6ae-db0b-4ba1-a637-ed5617892aff-sys\") pod \"perf-node-gather-daemonset-zgbfm\" (UID: \"3f87e6ae-db0b-4ba1-a637-ed5617892aff\") " pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:39.540882 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.540816 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3f87e6ae-db0b-4ba1-a637-ed5617892aff-podres\") pod \"perf-node-gather-daemonset-zgbfm\" (UID: \"3f87e6ae-db0b-4ba1-a637-ed5617892aff\") " pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:39.641974 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.641945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f87e6ae-db0b-4ba1-a637-ed5617892aff-sys\") pod \"perf-node-gather-daemonset-zgbfm\" (UID: \"3f87e6ae-db0b-4ba1-a637-ed5617892aff\") " pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:39.641974 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.641972 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3f87e6ae-db0b-4ba1-a637-ed5617892aff-podres\") pod \"perf-node-gather-daemonset-zgbfm\" (UID: \"3f87e6ae-db0b-4ba1-a637-ed5617892aff\") " pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:39.642132 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.642021 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3f87e6ae-db0b-4ba1-a637-ed5617892aff-proc\") pod \"perf-node-gather-daemonset-zgbfm\" (UID: \"3f87e6ae-db0b-4ba1-a637-ed5617892aff\") " pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:39.642132 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.642038 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f87e6ae-db0b-4ba1-a637-ed5617892aff-lib-modules\") pod \"perf-node-gather-daemonset-zgbfm\" (UID: \"3f87e6ae-db0b-4ba1-a637-ed5617892aff\") " pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:39.642132 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.642057 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f87e6ae-db0b-4ba1-a637-ed5617892aff-sys\") pod \"perf-node-gather-daemonset-zgbfm\" (UID: \"3f87e6ae-db0b-4ba1-a637-ed5617892aff\") " pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:39.642132 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.642064 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z74wr\" (UniqueName: \"kubernetes.io/projected/3f87e6ae-db0b-4ba1-a637-ed5617892aff-kube-api-access-z74wr\") pod \"perf-node-gather-daemonset-zgbfm\" (UID: \"3f87e6ae-db0b-4ba1-a637-ed5617892aff\") " pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:39.642132 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.642125 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3f87e6ae-db0b-4ba1-a637-ed5617892aff-proc\") pod \"perf-node-gather-daemonset-zgbfm\" (UID: \"3f87e6ae-db0b-4ba1-a637-ed5617892aff\") " pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:39.642381 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.642200 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3f87e6ae-db0b-4ba1-a637-ed5617892aff-podres\") pod \"perf-node-gather-daemonset-zgbfm\" (UID: \"3f87e6ae-db0b-4ba1-a637-ed5617892aff\") " pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:39.642381 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.642227 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f87e6ae-db0b-4ba1-a637-ed5617892aff-lib-modules\") pod \"perf-node-gather-daemonset-zgbfm\" (UID: \"3f87e6ae-db0b-4ba1-a637-ed5617892aff\") " pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:39.651664 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.651644 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z74wr\" (UniqueName: \"kubernetes.io/projected/3f87e6ae-db0b-4ba1-a637-ed5617892aff-kube-api-access-z74wr\") pod \"perf-node-gather-daemonset-zgbfm\" (UID: \"3f87e6ae-db0b-4ba1-a637-ed5617892aff\") " pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:39.752950 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.752887 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-2689n_e37d004a-3684-4ca0-ba67-c0da1a62496a/download-server/0.log" Sep 09 21:09:39.789984 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:39.789966 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:40.062751 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:40.062733 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm"] Sep 09 21:09:40.064952 ip-10-0-139-83 kubenswrapper[2562]: W0909 21:09:40.064921 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3f87e6ae_db0b_4ba1_a637_ed5617892aff.slice/crio-cb0c28fc3148ee59791a2ff400bb58bd9316446457f8772fc9f1077191f09ce3 WatchSource:0}: Error finding container cb0c28fc3148ee59791a2ff400bb58bd9316446457f8772fc9f1077191f09ce3: Status 404 returned error can't find the container with id cb0c28fc3148ee59791a2ff400bb58bd9316446457f8772fc9f1077191f09ce3 Sep 09 21:09:40.580748 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:40.580708 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" event={"ID":"3f87e6ae-db0b-4ba1-a637-ed5617892aff","Type":"ContainerStarted","Data":"91ab6a74615a6afc4565cf4cd9a5a7c4db0b7a85c777cad552719875563d96c7"} Sep 09 21:09:40.581134 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:40.580760 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" event={"ID":"3f87e6ae-db0b-4ba1-a637-ed5617892aff","Type":"ContainerStarted","Data":"cb0c28fc3148ee59791a2ff400bb58bd9316446457f8772fc9f1077191f09ce3"} Sep 09 21:09:40.581134 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:40.580790 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:40.599898 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:40.599850 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" podStartSLOduration=1.599836166 podStartE2EDuration="1.599836166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:09:40.597799656 +0000 UTC m=+1157.436998785" watchObservedRunningTime="2026-09-09 21:09:40.599836166 +0000 UTC m=+1157.439035295" Sep 09 21:09:40.633512 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:40.633486 2562 scope.go:117] "RemoveContainer" containerID="1bda35652734bcf189bf1490061e825a58793938600b769940636892a501ab91" Sep 09 21:09:40.633685 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:09:40.633666 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:09:41.631581 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:41.631553 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8wlxq_df144005-7adc-4de3-8c5b-37584f6ba534/dns/0.log" Sep 09 21:09:41.656646 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:41.656609 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8wlxq_df144005-7adc-4de3-8c5b-37584f6ba534/kube-rbac-proxy/0.log" Sep 09 21:09:41.790542 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:41.790497 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mvftd_d4d297c3-08db-42da-8621-b9b3d6ff8b9f/dns-node-resolver/0.log" Sep 09 21:09:42.340008 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:42.339978 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7d956956fc-bjp2t_f9f23ca0-784b-4f62-abea-cf8ac9060dad/registry/0.log" Sep 09 21:09:42.408233 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:42.408206 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sgvv4_51ddcfe5-9a4d-4fe5-96a8-ea3ffbf63f93/node-ca/0.log" Sep 09 21:09:43.368041 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:43.367994 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4_d64d0526-7ac1-4363-8fa9-b10cb0e9d334/istio-proxy/0.log" Sep 09 21:09:43.430603 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:43.430569 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-s7g9g_2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c/discovery/0.log" Sep 09 21:09:43.450775 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:43.450748 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55f8b7786b-7rk56_ba348cb1-cd13-4c92-854e-b059cd5612c6/kube-auth-proxy/0.log" Sep 09 21:09:43.532601 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:43.532571 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hcgf8_ac750ad1-5a58-43cb-b19a-53bf7c4ed6af/payload-pre-processing/0.log" Sep 09 21:09:43.559627 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:43.559605 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-2pfrd_142967f1-89cf-48be-877a-22461c043ada/payload-processing/0.log" Sep 09 21:09:44.233907 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:44.233869 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-crv44_b43b6a10-f027-4a3c-8e2a-fe42dfb285d4/serve-healthcheck-canary/0.log" Sep 09 21:09:44.718395 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:44.718371 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-2mxsn_5879ea93-d3c6-4607-a910-536119156bcf/insights-operator/1.log" Sep 09 21:09:44.718918 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:44.718899 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-2mxsn_5879ea93-d3c6-4607-a910-536119156bcf/insights-operator/0.log" Sep 09 21:09:44.817382 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:44.817352 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dct65_42671ab6-cd6a-4a2d-8494-828a140848f6/kube-rbac-proxy/0.log" Sep 09 21:09:44.853770 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:44.853749 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dct65_42671ab6-cd6a-4a2d-8494-828a140848f6/exporter/0.log" Sep 09 21:09:44.877546 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:44.877519 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dct65_42671ab6-cd6a-4a2d-8494-828a140848f6/extractor/0.log" Sep 09 21:09:46.596830 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:46.596804 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5djh2/perf-node-gather-daemonset-zgbfm" Sep 09 21:09:46.633668 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:46.633643 2562 scope.go:117] "RemoveContainer" containerID="70871f21d63390578a02ab75ed31340149aafb9c6e22a5d6735145cf3c18d800" Sep 09 21:09:46.633820 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:09:46.633808 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6_llm(73633935-aa97-47bf-b046-3077a14e5392)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-lr8x6" podUID="73633935-aa97-47bf-b046-3077a14e5392" Sep 09 21:09:47.294960 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:47.294926 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-8trts_3b65b188-5a8e-4307-9d22-4ffd70c6b7c6/server/0.log" Sep 09 21:09:47.380093 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:47.380061 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-ckv66_8d78d46f-4740-404e-984e-f647488edb65/manager/0.log" Sep 09 21:09:47.426449 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:47.426393 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cf9547dff-glg6x_0178f901-e8d9-42b6-b07a-662dadc9ace3/manager/0.log" Sep 09 21:09:47.460811 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:47.460787 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-cf9547dff-glg6x_0178f901-e8d9-42b6-b07a-662dadc9ace3/copy-manifests/0.log" Sep 09 21:09:49.953687 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:49.953643 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4_d64d0526-7ac1-4363-8fa9-b10cb0e9d334/istio-proxy/0.log" Sep 09 21:09:50.006098 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:50.006070 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-s7g9g_2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c/discovery/0.log" Sep 09 21:09:50.016792 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:50.016764 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55f8b7786b-7rk56_ba348cb1-cd13-4c92-854e-b059cd5612c6/kube-auth-proxy/0.log" Sep 09 21:09:50.072458 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:50.072424 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hcgf8_ac750ad1-5a58-43cb-b19a-53bf7c4ed6af/payload-pre-processing/0.log" Sep 09 21:09:50.090450 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:50.090427 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-2pfrd_142967f1-89cf-48be-877a-22461c043ada/payload-processing/0.log" Sep 09 21:09:51.241948 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:51.241918 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-z7ck4_d64d0526-7ac1-4363-8fa9-b10cb0e9d334/istio-proxy/0.log" Sep 09 21:09:51.306697 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:51.306670 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-s7g9g_2d14b124-b5c8-4b6e-a8a1-f6e20a26db4c/discovery/0.log" Sep 09 21:09:51.324048 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:51.324026 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55f8b7786b-7rk56_ba348cb1-cd13-4c92-854e-b059cd5612c6/kube-auth-proxy/0.log" Sep 09 21:09:51.414148 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:51.414120 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hcgf8_ac750ad1-5a58-43cb-b19a-53bf7c4ed6af/payload-pre-processing/0.log" Sep 09 21:09:51.447321 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:51.447256 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-2pfrd_142967f1-89cf-48be-877a-22461c043ada/payload-processing/0.log" Sep 09 21:09:54.226799 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:54.226773 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-vtlkg_15edbcc0-0649-4e9f-be70-1dcb3f2e0c61/migrator/0.log" Sep 09 21:09:54.248220 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:54.248195 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-vtlkg_15edbcc0-0649-4e9f-be70-1dcb3f2e0c61/graceful-termination/0.log" Sep 09 21:09:54.633523 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:54.633436 2562 scope.go:117] "RemoveContainer" containerID="1bda35652734bcf189bf1490061e825a58793938600b769940636892a501ab91" Sep 09 21:09:54.633700 ip-10-0-139-83 kubenswrapper[2562]: E0909 21:09:54.633672 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-7w6gs_llm(cc77e860-7a77-4800-83d3-42083f1a952b)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-7w6gs" podUID="cc77e860-7a77-4800-83d3-42083f1a952b" Sep 09 21:09:55.788738 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:55.788700 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prbnq_e26cba9c-5ba0-44c8-9d41-c63dad722bbe/kube-multus-additional-cni-plugins/0.log" Sep 09 21:09:55.812304 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:55.812280 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prbnq_e26cba9c-5ba0-44c8-9d41-c63dad722bbe/egress-router-binary-copy/0.log" Sep 09 21:09:55.835952 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:55.835923 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prbnq_e26cba9c-5ba0-44c8-9d41-c63dad722bbe/cni-plugins/0.log" Sep 09 21:09:55.859402 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:55.859377 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prbnq_e26cba9c-5ba0-44c8-9d41-c63dad722bbe/bond-cni-plugin/0.log" Sep 09 21:09:55.880872 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:55.880821 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prbnq_e26cba9c-5ba0-44c8-9d41-c63dad722bbe/routeoverride-cni/0.log" Sep 09 21:09:55.905164 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:55.905135 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prbnq_e26cba9c-5ba0-44c8-9d41-c63dad722bbe/whereabouts-cni-bincopy/0.log" Sep 09 21:09:55.929729 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:55.929703 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prbnq_e26cba9c-5ba0-44c8-9d41-c63dad722bbe/whereabouts-cni/0.log" Sep 09 21:09:56.145138 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:56.145074 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-krlkq_46ffa1a9-4a22-4274-b366-30bb80a2fa85/kube-multus/0.log" Sep 09 21:09:56.311534 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:56.311507 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lqd29_ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5/network-metrics-daemon/0.log" Sep 09 21:09:56.332282 ip-10-0-139-83 kubenswrapper[2562]: I0909 21:09:56.332256 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lqd29_ef398d96-b1a1-4b61-b0d0-1a4880fb3fb5/kube-rbac-proxy/0.log"