Sep 18 23:56:29.537829 ip-10-0-129-146 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 18 23:56:29.537843 ip-10-0-129-146 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 18 23:56:29.537851 ip-10-0-129-146 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 18 23:56:29.538109 ip-10-0-129-146 systemd[1]: Failed to start Kubernetes Kubelet. Sep 18 23:56:39.189592 ip-10-0-129-146 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Sep 18 23:56:39.189611 ip-10-0-129-146 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 3b86f55563104e3599954ac7af8116ae -- Sep 18 23:58:26.593652 ip-10-0-129-146 systemd[1]: Starting Kubernetes Kubelet... Sep 18 23:58:27.026495 ip-10-0-129-146 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 23:58:27.026495 ip-10-0-129-146 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 18 23:58:27.026495 ip-10-0-129-146 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 23:58:27.026495 ip-10-0-129-146 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 18 23:58:27.026495 ip-10-0-129-146 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 23:58:27.028238 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.028120 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 18 23:58:27.031543 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031528 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 23:58:27.031543 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031543 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031548 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031551 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031554 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031557 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031560 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031563 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031566 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031569 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031572 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031574 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031582 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031585 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031588 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031590 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031593 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031595 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031598 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031601 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031604 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 23:58:27.031604 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031607 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031611 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031616 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031619 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031622 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031625 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031627 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031630 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031633 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031635 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031638 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031640 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031643 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031646 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031648 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031651 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031654 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031656 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031659 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031662 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 23:58:27.032086 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031664 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031667 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031669 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031672 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031675 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031678 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031681 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031683 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031686 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031688 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031691 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031693 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031696 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031698 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031701 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031703 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031706 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031709 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031711 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 23:58:27.032619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031715 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031720 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031724 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031727 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031731 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031734 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031737 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031740 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031743 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031746 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031748 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031751 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031753 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031756 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031759 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031762 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031765 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031768 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031771 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 23:58:27.033083 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031773 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031776 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031779 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031781 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031784 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031787 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031790 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031792 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031795 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031798 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031800 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031803 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031806 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031808 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031811 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031814 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031817 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031819 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031822 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.031825 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 23:58:27.033558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032504 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032511 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032515 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032518 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032521 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032524 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032527 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032529 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032532 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032536 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032539 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032541 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032544 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032547 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032550 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032552 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032555 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032558 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032561 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 23:58:27.034063 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032563 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032566 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032569 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032572 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032574 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032577 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032579 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032582 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032585 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032587 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032590 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032593 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032596 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032598 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032601 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032604 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032606 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032609 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032612 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032616 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 23:58:27.034549 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032619 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032622 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032627 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032631 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032634 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032637 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032640 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032643 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032646 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032649 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032652 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032655 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032657 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032661 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032663 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032667 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032671 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032674 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032677 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 23:58:27.035069 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032680 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032683 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032685 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032688 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032691 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032693 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032696 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032699 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032702 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032704 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032707 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032710 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032713 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032716 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032719 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032722 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032724 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032727 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032731 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032733 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 23:58:27.035552 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032736 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032739 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032741 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032744 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032746 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032749 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032752 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032754 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032757 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032760 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032763 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032765 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032768 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032770 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032773 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032776 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032778 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032781 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032784 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032787 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 23:58:27.036047 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.032790 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033149 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033153 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033156 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033159 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033162 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033164 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033167 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033170 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033174 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033176 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033179 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033182 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033184 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033187 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033190 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033193 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033196 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033198 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033201 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 23:58:27.036556 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033204 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033206 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033209 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033212 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033214 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033217 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033220 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033222 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033225 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033227 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033230 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033233 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033236 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033238 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033240 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033243 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033246 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033248 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033251 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 23:58:27.037096 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033254 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033257 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033260 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033262 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033265 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033267 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033270 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033272 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033275 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033278 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033280 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033283 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033285 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033288 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033291 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033294 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033296 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033299 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033301 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 23:58:27.037577 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033304 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033309 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033313 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033316 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033320 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033323 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033326 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033329 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033331 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033334 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033337 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033339 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033344 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033347 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033350 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033352 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033355 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033358 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033360 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 23:58:27.038136 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033363 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033365 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033384 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033387 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033392 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033394 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033400 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033404 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033407 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033410 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033413 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033416 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033419 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033421 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033424 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033427 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033429 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033432 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033434 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 23:58:27.038616 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033437 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033440 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033443 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.033446 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033520 2568 flags.go:64] FLAG: --address="0.0.0.0" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033553 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033561 2568 flags.go:64] FLAG: --anonymous-auth="true" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033566 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033570 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033573 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033577 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033582 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033585 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033588 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033591 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033594 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033597 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033600 2568 flags.go:64] FLAG: --cgroup-root="" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033603 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033606 2568 flags.go:64] FLAG: --client-ca-file="" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033609 2568 flags.go:64] FLAG: --cloud-provider="external" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033612 2568 flags.go:64] FLAG: --cluster-dns="[]" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033616 2568 flags.go:64] FLAG: --cluster-domain="" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033618 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 18 23:58:27.039084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033621 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033624 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033627 2568 flags.go:64] FLAG: --container-log-max-files="5" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033631 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033634 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033637 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033650 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033653 2568 flags.go:64] FLAG: --contention-profiling="false" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033656 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033659 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033663 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033666 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033670 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033672 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033679 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033682 2568 flags.go:64] FLAG: --enable-load-reader="false" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033684 2568 flags.go:64] FLAG: --enable-server="true" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033687 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033692 2568 flags.go:64] FLAG: --event-burst="100" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033695 2568 flags.go:64] FLAG: --event-qps="50" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033698 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033701 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033704 2568 flags.go:64] FLAG: --eviction-hard="" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033707 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 18 23:58:27.039679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033710 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033713 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033716 2568 flags.go:64] FLAG: --eviction-soft="" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033720 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033722 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033725 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033728 2568 flags.go:64] FLAG: --experimental-mounter-path="" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033731 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033734 2568 flags.go:64] FLAG: --fail-swap-on="true" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033737 2568 flags.go:64] FLAG: --feature-gates="" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033740 2568 flags.go:64] FLAG: --file-check-frequency="20s" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033743 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033746 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033749 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033752 2568 flags.go:64] FLAG: --healthz-port="10248" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033755 2568 flags.go:64] FLAG: --help="false" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033758 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033761 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033764 2568 flags.go:64] FLAG: --http-check-frequency="20s" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033768 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033771 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033774 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033778 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 18 23:58:27.040273 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033781 2568 flags.go:64] FLAG: --image-service-endpoint="" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033784 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033787 2568 flags.go:64] FLAG: --kube-api-burst="100" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033789 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033793 2568 flags.go:64] FLAG: --kube-api-qps="50" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033796 2568 flags.go:64] FLAG: --kube-reserved="" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033798 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033801 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033804 2568 flags.go:64] FLAG: --kubelet-cgroups="" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033807 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033810 2568 flags.go:64] FLAG: --lock-file="" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033813 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033816 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033819 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033824 2568 flags.go:64] FLAG: --log-json-split-stream="false" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033827 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033830 2568 flags.go:64] FLAG: --log-text-split-stream="false" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033832 2568 flags.go:64] FLAG: --logging-format="text" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033835 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033838 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033841 2568 flags.go:64] FLAG: --manifest-url="" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033844 2568 flags.go:64] FLAG: --manifest-url-header="" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033848 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033851 2568 flags.go:64] FLAG: --max-open-files="1000000" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033855 2568 flags.go:64] FLAG: --max-pods="110" Sep 18 23:58:27.040896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033858 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033861 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033863 2568 flags.go:64] FLAG: --memory-manager-policy="None" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033866 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033870 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033873 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033877 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033884 2568 flags.go:64] FLAG: --node-status-max-images="50" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033887 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033890 2568 flags.go:64] FLAG: --oom-score-adj="-999" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033893 2568 flags.go:64] FLAG: --pod-cidr="" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033896 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033901 2568 flags.go:64] FLAG: --pod-manifest-path="" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033904 2568 flags.go:64] FLAG: --pod-max-pids="-1" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033907 2568 flags.go:64] FLAG: --pods-per-core="0" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033910 2568 flags.go:64] FLAG: --port="10250" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033913 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033916 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01c216085d45e80bc" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033919 2568 flags.go:64] FLAG: --qos-reserved="" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033922 2568 flags.go:64] FLAG: --read-only-port="10255" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033924 2568 flags.go:64] FLAG: --register-node="true" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033927 2568 flags.go:64] FLAG: --register-with-taints="" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033931 2568 flags.go:64] FLAG: --registry-burst="10" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033934 2568 flags.go:64] FLAG: --registry-qps="5" Sep 18 23:58:27.041549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033937 2568 flags.go:64] FLAG: --reserved-cpus="" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033940 2568 flags.go:64] FLAG: --reserved-memory="" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033943 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033946 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033949 2568 flags.go:64] FLAG: --rotate-certificates="false" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033952 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033955 2568 flags.go:64] FLAG: --runonce="false" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033958 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033961 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033964 2568 flags.go:64] FLAG: --seccomp-default="false" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033967 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033970 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033973 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033976 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033980 2568 flags.go:64] FLAG: --storage-driver-password="root" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033983 2568 flags.go:64] FLAG: --storage-driver-secure="false" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033986 2568 flags.go:64] FLAG: --storage-driver-table="stats" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033989 2568 flags.go:64] FLAG: --storage-driver-user="root" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033992 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033995 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.033997 2568 flags.go:64] FLAG: --system-cgroups="" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.034000 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.034023 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.034027 2568 flags.go:64] FLAG: --tls-cert-file="" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.034031 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 18 23:58:27.042153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.034036 2568 flags.go:64] FLAG: --tls-min-version="" Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.034039 2568 flags.go:64] FLAG: --tls-private-key-file="" Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.034042 2568 flags.go:64] FLAG: --topology-manager-policy="none" Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.034045 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.034048 2568 flags.go:64] FLAG: --topology-manager-scope="container" Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.034051 2568 flags.go:64] FLAG: --v="2" Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.034055 2568 flags.go:64] FLAG: --version="false" Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.034059 2568 flags.go:64] FLAG: --vmodule="" Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.034063 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.034067 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034180 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034184 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034188 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034191 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034195 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034198 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034200 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034203 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034206 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034208 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034211 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034214 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 23:58:27.042820 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034218 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034221 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034224 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034227 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034229 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034232 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034234 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034237 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034240 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034243 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034247 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034250 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034252 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034255 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034257 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034260 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034263 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034266 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034268 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 23:58:27.043380 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034271 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034273 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034276 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034279 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034281 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034284 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034286 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034289 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034291 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034294 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034297 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034299 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034302 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034307 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034310 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034315 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034319 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034322 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034324 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034328 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 23:58:27.043851 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034331 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034334 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034337 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034341 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034343 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034346 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034349 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034352 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034354 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034357 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034360 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034362 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034365 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034381 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034384 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034386 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034391 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034394 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034397 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 23:58:27.044342 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034400 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034402 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034405 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034407 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034410 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034413 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034417 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034425 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034428 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034431 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034434 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034437 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034440 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034443 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034446 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034449 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034453 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034456 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034459 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 23:58:27.044843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034462 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 23:58:27.045633 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034464 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 23:58:27.045633 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034467 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 23:58:27.045633 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034470 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 23:58:27.045633 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034472 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 23:58:27.045633 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034475 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 23:58:27.045633 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034478 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 23:58:27.045633 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034481 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 23:58:27.045633 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034483 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 23:58:27.045633 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.034486 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 23:58:27.045633 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.035350 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 23:58:27.045633 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.043654 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 18 23:58:27.045633 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.043670 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 18 23:58:27.045633 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.044725 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 18 23:58:27.045633 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.044735 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 18 23:58:27.045633 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045025 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045038 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045044 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045049 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045054 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045059 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045065 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045069 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045073 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045084 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045088 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045092 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045096 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045103 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045108 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045113 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045117 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045122 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045126 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045131 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 23:58:27.046130 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045135 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045144 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045148 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045152 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045157 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045161 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045165 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045170 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045174 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045180 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045184 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045189 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045193 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045201 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045206 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045210 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045214 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045218 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045223 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045227 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 23:58:27.046645 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045232 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045236 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045240 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045244 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045249 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045252 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045261 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045265 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045270 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045274 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045279 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045283 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045287 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045291 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045295 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045299 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045303 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045307 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045312 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045321 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 23:58:27.047128 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045326 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045331 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045336 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045340 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045344 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045348 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045352 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045359 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045365 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045386 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045390 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045402 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045408 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045413 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045417 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045422 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045426 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045430 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045434 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 23:58:27.047636 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045438 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045442 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045446 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045450 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045459 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045463 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045467 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045472 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045476 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045480 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045484 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045489 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045493 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045498 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045503 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045508 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045512 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045521 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045526 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 23:58:27.048135 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.045531 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 23:58:27.048619 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.045539 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 23:58:27.048619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046088 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 23:58:27.048619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046097 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 23:58:27.048619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046101 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 23:58:27.048619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046105 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 23:58:27.048619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046109 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 23:58:27.048619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046112 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 23:58:27.048619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046115 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 23:58:27.048619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046118 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 23:58:27.048619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046121 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 23:58:27.048619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046124 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 23:58:27.048619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046127 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 23:58:27.048619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046129 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 23:58:27.048619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046132 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 23:58:27.048619 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046135 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046138 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046141 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046144 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046147 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046150 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046152 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046155 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046158 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046160 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046163 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046166 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046169 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046172 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046175 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046179 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046184 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046187 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046190 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 23:58:27.048984 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046193 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046197 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046200 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046204 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046207 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046209 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046212 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046216 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046220 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046223 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046226 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046228 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046231 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046234 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046236 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046239 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046242 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046244 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046247 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046249 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 23:58:27.049558 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046252 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046255 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046258 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046261 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046263 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046267 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046270 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046272 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046275 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046278 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046280 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046284 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046287 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046289 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046292 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046295 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046297 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046300 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046303 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046305 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 23:58:27.050062 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046308 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046310 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046313 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046316 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046318 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046321 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046324 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046327 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046330 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046333 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046335 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046338 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046341 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046343 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046346 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046349 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046352 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046355 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046357 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046361 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 23:58:27.050555 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046363 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 23:58:27.051072 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046366 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 23:58:27.051072 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046385 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 23:58:27.051072 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046388 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 23:58:27.051072 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046392 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 23:58:27.051072 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046396 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 23:58:27.051072 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:27.046399 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 23:58:27.051072 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.046405 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 23:58:27.051072 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.046523 2568 server.go:962] "Client rotation is on, will bootstrap in background" Sep 18 23:58:27.051072 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.049850 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 18 23:58:27.051072 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.050831 2568 server.go:1019] "Starting client certificate rotation" Sep 18 23:58:27.051072 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.050909 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 18 23:58:27.051072 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.050952 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 18 23:58:27.061926 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.056285 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 18 23:58:27.061926 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.058113 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 18 23:58:27.074495 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.074315 2568 log.go:25] "Validated CRI v1 runtime API" Sep 18 23:58:27.077899 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.077878 2568 log.go:25] "Validated CRI v1 image API" Sep 18 23:58:27.078811 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.078792 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 18 23:58:27.082983 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.082960 2568 fs.go:135] Filesystem UUIDs: map[4f2efcfd-3885-4625-918d-5f5bbef51b94:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 aa8ddfba-6cfc-4d0f-ab64-21dc49697ed1:/dev/nvme0n1p3] Sep 18 23:58:27.083065 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.082979 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 18 23:58:27.085706 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.085686 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 18 23:58:27.088441 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.088315 2568 manager.go:216] Machine: {Timestamp:2026-09-18 23:58:27.086859828 +0000 UTC m=+0.380876838 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3113498 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec242c0f1c6b0b0986caa571e95649a5 SystemUUID:ec242c0f-1c6b-0b09-86ca-a571e95649a5 BootID:3b86f555-6310-4e35-9995-4ac7af8116ae Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f9:7d:15:63 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f9:7d:15:63 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:76:35:76:5d:3f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 18 23:58:27.088441 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.088440 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 18 23:58:27.088542 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.088526 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 18 23:58:27.090565 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.090537 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 18 23:58:27.090701 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.090568 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-146.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 18 23:58:27.091421 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.091411 2568 topology_manager.go:138] "Creating topology manager with none policy" Sep 18 23:58:27.091453 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.091424 2568 container_manager_linux.go:309] "Creating device plugin manager" Sep 18 23:58:27.091453 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.091432 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 18 23:58:27.093051 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.093041 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 18 23:58:27.094871 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.094861 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 18 23:58:27.100074 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.100061 2568 state_mem.go:36] "Initialized new in-memory state store" Sep 18 23:58:27.100192 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.100183 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 18 23:58:27.100227 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.100216 2568 kubelet.go:486] "Attempting to sync node with API server" Sep 18 23:58:27.100254 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.100227 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 18 23:58:27.100254 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.100244 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 18 23:58:27.100254 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.100253 2568 kubelet.go:398] "Adding apiserver pod source" Sep 18 23:58:27.100336 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.100262 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 18 23:58:27.100691 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.100676 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 18 23:58:27.100756 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.100695 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 18 23:58:27.103183 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.103153 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bt7vr" Sep 18 23:58:27.104246 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.104154 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1" Sep 18 23:58:27.107575 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.107559 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 18 23:58:27.107644 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.107590 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 18 23:58:27.107755 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.107744 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 18 23:58:27.107793 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.107769 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 18 23:58:27.107793 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.107777 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 18 23:58:27.107793 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.107782 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 18 23:58:27.107793 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.107790 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 18 23:58:27.107905 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.107795 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 18 23:58:27.108411 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.108396 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 18 23:58:27.108411 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.108407 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 18 23:58:27.108411 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.108414 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 18 23:58:27.108547 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.108419 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 18 23:58:27.108547 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.108428 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 18 23:58:27.108547 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.108436 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 18 23:58:27.109255 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.109241 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 18 23:58:27.109255 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.109251 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 18 23:58:27.111125 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.111110 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bt7vr" Sep 18 23:58:27.111822 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.111790 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 18 23:58:27.111822 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.111790 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-146.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 18 23:58:27.113281 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.113268 2568 server.go:1268] "Started kubelet" Sep 18 23:58:27.113401 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.113330 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 18 23:58:27.113450 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.113338 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 18 23:58:27.113486 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.113452 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 18 23:58:27.114113 ip-10-0-129-146 systemd[1]: Started Kubernetes Kubelet. Sep 18 23:58:27.116131 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.114868 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 18 23:58:27.116131 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.115495 2568 server.go:310] "Adding debug handlers to kubelet server" Sep 18 23:58:27.120494 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.120462 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 18 23:58:27.123794 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.123775 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 18 23:58:27.124257 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.124239 2568 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-146.ec2.internal" not found Sep 18 23:58:27.124463 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.124451 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 18 23:58:27.124583 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.124566 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-146.ec2.internal\" not found" Sep 18 23:58:27.125251 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.125237 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 18 23:58:27.125251 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.125239 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 18 23:58:27.125405 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.125261 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 18 23:58:27.125405 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.125392 2568 reconstruct.go:97] "Volume reconstruction finished" Sep 18 23:58:27.125405 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.125400 2568 reconciler.go:29] "Reconciler: start to sync state" Sep 18 23:58:27.125537 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.125454 2568 factory.go:55] Registering systemd factory Sep 18 23:58:27.125537 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.125471 2568 factory.go:223] Registration of the systemd container factory successfully Sep 18 23:58:27.125698 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.125677 2568 factory.go:153] Registering CRI-O factory Sep 18 23:58:27.125698 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.125691 2568 factory.go:223] Registration of the crio container factory successfully Sep 18 23:58:27.125802 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.125733 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 18 23:58:27.125802 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.125747 2568 factory.go:103] Registering Raw factory Sep 18 23:58:27.125802 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.125757 2568 manager.go:1196] Started watching for new ooms in manager Sep 18 23:58:27.126626 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.126608 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 23:58:27.126942 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.126928 2568 manager.go:349] Starting recovery of all containers Sep 18 23:58:27.128980 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.128958 2568 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-146.ec2.internal\" not found" node="ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.138299 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.137460 2568 manager.go:354] Recovery completed Sep 18 23:58:27.139968 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.139945 2568 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Sep 18 23:58:27.141835 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.141817 2568 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-146.ec2.internal" not found Sep 18 23:58:27.143163 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.143151 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 23:58:27.145052 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.145036 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasSufficientMemory" Sep 18 23:58:27.145107 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.145070 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 23:58:27.145107 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.145082 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasSufficientPID" Sep 18 23:58:27.145596 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.145582 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 18 23:58:27.145596 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.145591 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 23:58:27.145683 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.145597 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 18 23:58:27.145683 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.145616 2568 state_mem.go:36] "Initialized new in-memory state store" Sep 18 23:58:27.146255 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.146241 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasSufficientMemory" Sep 18 23:58:27.146297 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.146265 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 23:58:27.146297 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.146276 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasSufficientPID" Sep 18 23:58:27.147634 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.147622 2568 policy_none.go:49] "None policy: Start" Sep 18 23:58:27.147673 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.147638 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 18 23:58:27.147673 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.147648 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 18 23:58:27.149480 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.149468 2568 policy_none.go:47] "Start" Sep 18 23:58:27.190024 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.190007 2568 manager.go:337] "Starting Device Plugin manager" Sep 18 23:58:27.210847 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.190053 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 18 23:58:27.210847 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.190068 2568 server.go:87] "Starting device plugin registration server" Sep 18 23:58:27.210847 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.190325 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 18 23:58:27.210847 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.190336 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 18 23:58:27.210847 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.190504 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 18 23:58:27.210847 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.190590 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 18 23:58:27.210847 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.190600 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 18 23:58:27.210847 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.191043 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 18 23:58:27.210847 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.191086 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-146.ec2.internal\" not found" Sep 18 23:58:27.210847 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.197685 2568 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-129-146.ec2.internal" not found Sep 18 23:58:27.231867 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.231839 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 18 23:58:27.233172 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.233160 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 18 23:58:27.233238 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.233203 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 18 23:58:27.233238 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.233227 2568 kubelet.go:2448] "Starting kubelet main sync loop" Sep 18 23:58:27.233343 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.233263 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 18 23:58:27.235353 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.235334 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 23:58:27.291509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.291413 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 23:58:27.292396 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.292349 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasSufficientMemory" Sep 18 23:58:27.292396 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.292398 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 23:58:27.292521 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.292413 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasSufficientPID" Sep 18 23:58:27.292521 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.292438 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.300754 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.300737 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.300841 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.300764 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-146.ec2.internal\": node \"ip-10-0-129-146.ec2.internal\" not found" Sep 18 23:58:27.313969 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.313945 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-146.ec2.internal\" not found" Sep 18 23:58:27.334347 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.334325 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-146.ec2.internal"] Sep 18 23:58:27.334437 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.334426 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 23:58:27.335293 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.335277 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasSufficientMemory" Sep 18 23:58:27.335404 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.335304 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 23:58:27.335404 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.335314 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasSufficientPID" Sep 18 23:58:27.337520 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.337508 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 23:58:27.337651 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.337628 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.337689 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.337671 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 23:58:27.339110 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.339094 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasSufficientMemory" Sep 18 23:58:27.339184 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.339127 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 23:58:27.339184 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.339142 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasSufficientPID" Sep 18 23:58:27.339184 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.339147 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasSufficientMemory" Sep 18 23:58:27.339184 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.339163 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 23:58:27.339184 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.339172 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasSufficientPID" Sep 18 23:58:27.341360 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.341343 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.341448 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.341391 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 23:58:27.342445 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.342429 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasSufficientMemory" Sep 18 23:58:27.342511 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.342462 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 23:58:27.342511 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.342478 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeHasSufficientPID" Sep 18 23:58:27.366088 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.366068 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-146.ec2.internal\" not found" node="ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.370696 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.370682 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-146.ec2.internal\" not found" node="ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.414954 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.414932 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-146.ec2.internal\" not found" Sep 18 23:58:27.426832 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.426813 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bae7cc99672b5dbd116a90c47b4ad4c3-config\") pod \"kube-apiserver-proxy-ip-10-0-129-146.ec2.internal\" (UID: \"bae7cc99672b5dbd116a90c47b4ad4c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.426940 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.426837 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/03c6c038cb8cb442ed4c13464d8fdf16-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal\" (UID: \"03c6c038cb8cb442ed4c13464d8fdf16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.426940 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.426859 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03c6c038cb8cb442ed4c13464d8fdf16-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal\" (UID: \"03c6c038cb8cb442ed4c13464d8fdf16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.515730 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.515684 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-146.ec2.internal\" not found" Sep 18 23:58:27.527023 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.526997 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bae7cc99672b5dbd116a90c47b4ad4c3-config\") pod \"kube-apiserver-proxy-ip-10-0-129-146.ec2.internal\" (UID: \"bae7cc99672b5dbd116a90c47b4ad4c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.527124 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.527038 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/03c6c038cb8cb442ed4c13464d8fdf16-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal\" (UID: \"03c6c038cb8cb442ed4c13464d8fdf16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.527124 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.527057 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bae7cc99672b5dbd116a90c47b4ad4c3-config\") pod \"kube-apiserver-proxy-ip-10-0-129-146.ec2.internal\" (UID: \"bae7cc99672b5dbd116a90c47b4ad4c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.527124 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.527066 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03c6c038cb8cb442ed4c13464d8fdf16-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal\" (UID: \"03c6c038cb8cb442ed4c13464d8fdf16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.527124 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.527089 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/03c6c038cb8cb442ed4c13464d8fdf16-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal\" (UID: \"03c6c038cb8cb442ed4c13464d8fdf16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.527124 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.527095 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03c6c038cb8cb442ed4c13464d8fdf16-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal\" (UID: \"03c6c038cb8cb442ed4c13464d8fdf16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.616492 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.616424 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-146.ec2.internal\" not found" Sep 18 23:58:27.670879 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.670857 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.674026 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:27.674010 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-146.ec2.internal" Sep 18 23:58:27.716883 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.716833 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-146.ec2.internal\" not found" Sep 18 23:58:27.817449 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.817418 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-146.ec2.internal\" not found" Sep 18 23:58:27.917966 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:27.917890 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-146.ec2.internal\" not found" Sep 18 23:58:28.018467 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:28.018430 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-146.ec2.internal\" not found" Sep 18 23:58:28.050875 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:28.050849 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 18 23:58:28.051337 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:28.051020 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 18 23:58:28.051337 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:28.051044 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 18 23:58:28.114385 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:28.114320 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 23:53:29 +0000 UTC" deadline="2028-02-18 22:39:24.746647544 +0000 UTC" Sep 18 23:58:28.114385 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:28.114355 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12430h40m56.632295713s" Sep 18 23:58:28.119494 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:28.119472 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-146.ec2.internal\" not found" Sep 18 23:58:28.210020 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:28.209989 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c6c038cb8cb442ed4c13464d8fdf16.slice/crio-f6b9530ba438f2b6cc29f8c1642a4a9b3c3ced934186c1776efb05d35d5e3b2b WatchSource:0}: Error finding container f6b9530ba438f2b6cc29f8c1642a4a9b3c3ced934186c1776efb05d35d5e3b2b: Status 404 returned error can't find the container with id f6b9530ba438f2b6cc29f8c1642a4a9b3c3ced934186c1776efb05d35d5e3b2b Sep 18 23:58:28.210343 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:28.210328 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbae7cc99672b5dbd116a90c47b4ad4c3.slice/crio-75b87f9e5b94ef4543f6a5ca42c07877755f0a1fb1782cdb552dbf721b06080b WatchSource:0}: Error finding container 75b87f9e5b94ef4543f6a5ca42c07877755f0a1fb1782cdb552dbf721b06080b: Status 404 returned error can't find the container with id 75b87f9e5b94ef4543f6a5ca42c07877755f0a1fb1782cdb552dbf721b06080b Sep 18 23:58:28.215614 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:28.215600 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 23:58:28.220445 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:28.220426 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-146.ec2.internal\" not found" Sep 18 23:58:28.222991 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:28.222975 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 23:58:28.224732 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:28.224717 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal" Sep 18 23:58:28.234295 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:28.234280 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 18 23:58:28.236167 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:28.236151 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-146.ec2.internal" Sep 18 23:58:28.236279 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:28.236234 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal" event={"ID":"03c6c038cb8cb442ed4c13464d8fdf16","Type":"ContainerStarted","Data":"f6b9530ba438f2b6cc29f8c1642a4a9b3c3ced934186c1776efb05d35d5e3b2b"} Sep 18 23:58:28.237264 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:28.237248 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-146.ec2.internal" event={"ID":"bae7cc99672b5dbd116a90c47b4ad4c3","Type":"ContainerStarted","Data":"75b87f9e5b94ef4543f6a5ca42c07877755f0a1fb1782cdb552dbf721b06080b"} Sep 18 23:58:28.241722 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:28.241709 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 18 23:58:28.394174 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:28.394145 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 23:58:31.242081 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.241858 2568 apiserver.go:52] "Watching apiserver" Sep 18 23:58:31.253110 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.253082 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 18 23:58:31.253493 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.253463 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-146.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg","openshift-dns/node-resolver-frlfs","openshift-multus/multus-additional-cni-plugins-24d8b","openshift-multus/multus-wbld8","openshift-network-operator/iptables-alerter-r2q4h","kube-system/global-pull-secret-syncer-7nzpw","openshift-cluster-node-tuning-operator/tuned-9mgwp","openshift-image-registry/node-ca-2k8jv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal","openshift-multus/network-metrics-daemon-g2zzv","openshift-network-diagnostics/network-check-target-lpqlq","openshift-ovn-kubernetes/ovnkube-node-mmfk5","kube-system/konnectivity-agent-qsdhf"] Sep 18 23:58:31.255903 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.255888 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:31.255974 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:31.255952 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:58:31.258171 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.258153 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.260774 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.260749 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 18 23:58:31.260895 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.260842 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 18 23:58:31.260895 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.260871 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 18 23:58:31.261127 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.261112 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-frlfs" Sep 18 23:58:31.261656 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.261637 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 18 23:58:31.261656 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.261649 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 18 23:58:31.261774 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.261639 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 18 23:58:31.261774 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.261720 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-btxx4\"" Sep 18 23:58:31.263267 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.263247 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.263379 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.263355 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 18 23:58:31.263479 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.263455 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nnln5\"" Sep 18 23:58:31.264065 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.264044 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 18 23:58:31.265255 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.265063 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 18 23:58:31.265318 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.265300 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 18 23:58:31.265433 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.265415 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.265706 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.265688 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 18 23:58:31.265963 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.265948 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-q67ct\"" Sep 18 23:58:31.266296 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.266279 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 18 23:58:31.266402 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.266278 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 18 23:58:31.266529 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.266278 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 18 23:58:31.267653 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.267625 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 18 23:58:31.267760 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.267744 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r2q4h" Sep 18 23:58:31.267857 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.267849 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rj8nk\"" Sep 18 23:58:31.270047 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.270027 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kqppx\"" Sep 18 23:58:31.270145 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.270075 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 18 23:58:31.270211 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.270033 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 18 23:58:31.270325 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.270310 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 18 23:58:31.270415 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.270315 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2k8jv" Sep 18 23:58:31.272278 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.272254 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qsdhf" Sep 18 23:58:31.272577 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.272558 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 18 23:58:31.272665 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.272654 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 18 23:58:31.272743 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.272727 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zd2sm\"" Sep 18 23:58:31.272821 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.272751 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 18 23:58:31.274956 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.274522 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.275449 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.275392 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 18 23:58:31.275449 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.275426 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 18 23:58:31.275449 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.275441 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mlgcs\"" Sep 18 23:58:31.275639 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.275617 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfnqq\" (UniqueName: \"kubernetes.io/projected/6691ea3d-128e-47ee-b52b-2a48ca799dc8-kube-api-access-nfnqq\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.275761 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.275652 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-run-netns\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.275761 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.275681 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6523199f-3f99-4e7f-ac81-4462f800b35d-cni-binary-copy\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.275761 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.275706 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-multus-conf-dir\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.275950 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.275773 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6523199f-3f99-4e7f-ac81-4462f800b35d-multus-daemon-config\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.275950 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.275913 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blnjv\" (UniqueName: \"kubernetes.io/projected/b6aa6323-b7b5-4f5b-8426-904ec24f1d79-kube-api-access-blnjv\") pod \"iptables-alerter-r2q4h\" (UID: \"b6aa6323-b7b5-4f5b-8426-904ec24f1d79\") " pod="openshift-network-operator/iptables-alerter-r2q4h" Sep 18 23:58:31.275950 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.275943 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c29d9e92-fea6-47b8-b299-ec7ae98f3010-dbus\") pod \"global-pull-secret-syncer-7nzpw\" (UID: \"c29d9e92-fea6-47b8-b299-ec7ae98f3010\") " pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:31.276100 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.275981 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-run-netns\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.276100 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276006 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-cni-netd\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.276100 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276029 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-run-multus-certs\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.276100 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276052 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-etc-kubernetes\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.276100 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276075 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret\") pod \"global-pull-secret-syncer-7nzpw\" (UID: \"c29d9e92-fea6-47b8-b299-ec7ae98f3010\") " pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:31.276330 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276101 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.276330 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276126 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-hostroot\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.276330 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276154 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-system-cni-dir\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.276330 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276177 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh5kd\" (UniqueName: \"kubernetes.io/projected/6523199f-3f99-4e7f-ac81-4462f800b35d-kube-api-access-dh5kd\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.276330 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276199 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c29d9e92-fea6-47b8-b299-ec7ae98f3010-kubelet-config\") pod \"global-pull-secret-syncer-7nzpw\" (UID: \"c29d9e92-fea6-47b8-b299-ec7ae98f3010\") " pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:31.276330 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276222 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-slash\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.276330 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276252 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-log-socket\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.276330 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276286 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vmvg\" (UniqueName: \"kubernetes.io/projected/ad0b3adb-7a95-473f-b3c9-2ac12f522495-kube-api-access-4vmvg\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.276330 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276320 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fljtw\" (UniqueName: \"kubernetes.io/projected/1df5be81-ad93-46f5-9886-90e729bdc767-kube-api-access-fljtw\") pod \"node-resolver-frlfs\" (UID: \"1df5be81-ad93-46f5-9886-90e729bdc767\") " pod="openshift-dns/node-resolver-frlfs" Sep 18 23:58:31.276727 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276345 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6aa6323-b7b5-4f5b-8426-904ec24f1d79-host-slash\") pod \"iptables-alerter-r2q4h\" (UID: \"b6aa6323-b7b5-4f5b-8426-904ec24f1d79\") " pod="openshift-network-operator/iptables-alerter-r2q4h" Sep 18 23:58:31.276727 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276369 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-systemd-units\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.276727 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276391 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-run-systemd\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.276727 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276414 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-run-ovn-kubernetes\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.276727 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276443 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6691ea3d-128e-47ee-b52b-2a48ca799dc8-system-cni-dir\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.276727 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276468 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6691ea3d-128e-47ee-b52b-2a48ca799dc8-cnibin\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.276727 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276494 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6691ea3d-128e-47ee-b52b-2a48ca799dc8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.276727 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276519 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6691ea3d-128e-47ee-b52b-2a48ca799dc8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.276727 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276544 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-multus-cni-dir\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.276727 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276591 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-var-lib-openvswitch\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.276727 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276686 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-run-openvswitch\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.276727 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276706 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-node-log\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.276727 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276721 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-cni-bin\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.276727 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276735 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ad0b3adb-7a95-473f-b3c9-2ac12f522495-ovn-node-metrics-cert\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.277422 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276752 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-cnibin\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.277422 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276775 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-var-lib-cni-multus\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.277422 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276814 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 18 23:58:31.277422 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276816 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-var-lib-kubelet\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.277422 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.276997 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b6aa6323-b7b5-4f5b-8426-904ec24f1d79-iptables-alerter-script\") pod \"iptables-alerter-r2q4h\" (UID: \"b6aa6323-b7b5-4f5b-8426-904ec24f1d79\") " pod="openshift-network-operator/iptables-alerter-r2q4h" Sep 18 23:58:31.277422 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.277023 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-etc-openvswitch\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.277422 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.277355 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t94r9\"" Sep 18 23:58:31.277422 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.277049 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ad0b3adb-7a95-473f-b3c9-2ac12f522495-ovnkube-script-lib\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.277422 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.277416 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1df5be81-ad93-46f5-9886-90e729bdc767-hosts-file\") pod \"node-resolver-frlfs\" (UID: \"1df5be81-ad93-46f5-9886-90e729bdc767\") " pod="openshift-dns/node-resolver-frlfs" Sep 18 23:58:31.277824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.277446 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1df5be81-ad93-46f5-9886-90e729bdc767-tmp-dir\") pod \"node-resolver-frlfs\" (UID: \"1df5be81-ad93-46f5-9886-90e729bdc767\") " pod="openshift-dns/node-resolver-frlfs" Sep 18 23:58:31.277824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.277462 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.277824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.277470 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-var-lib-cni-bin\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.277824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.277501 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-multus-socket-dir-parent\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.277824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.277528 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-run-k8s-cni-cncf-io\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.277824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.277545 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-kubelet\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.277824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.277555 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 18 23:58:31.277824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.277532 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 18 23:58:31.277824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.277558 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ad0b3adb-7a95-473f-b3c9-2ac12f522495-ovnkube-config\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.277824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.277691 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-run-ovn\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.278773 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.278511 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ad0b3adb-7a95-473f-b3c9-2ac12f522495-env-overrides\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.278773 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.278563 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6691ea3d-128e-47ee-b52b-2a48ca799dc8-cni-binary-copy\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.278773 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.278595 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6691ea3d-128e-47ee-b52b-2a48ca799dc8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.279840 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.279819 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 18 23:58:31.280695 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.280677 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 18 23:58:31.281533 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.281516 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:31.281650 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:31.281607 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2zzv" podUID="0092b756-14b9-4468-b25e-47a279c0e9a1" Sep 18 23:58:31.283824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.282363 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5gl2h\"" Sep 18 23:58:31.283824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.283187 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 18 23:58:31.284272 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.284249 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:31.284429 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:31.284325 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpqlq" podUID="1c03ca88-a515-4cf1-b753-c8e93034644b" Sep 18 23:58:31.310073 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.310048 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tmc9j" Sep 18 23:58:31.315258 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.315136 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tmc9j" Sep 18 23:58:31.367082 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.367059 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 18 23:58:31.378719 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.378699 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-run-netns\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.378875 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.378724 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6523199f-3f99-4e7f-ac81-4462f800b35d-cni-binary-copy\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.378875 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.378739 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-run-netns\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.378875 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.378757 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-socket-dir\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.378875 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.378837 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-run-netns\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.378875 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.378853 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-run-netns\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.379132 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.378876 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq9b5\" (UniqueName: \"kubernetes.io/projected/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-kube-api-access-cq9b5\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.379132 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.378902 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/109db1fc-2eed-4932-b461-7acbf67f88e7-serviceca\") pod \"node-ca-2k8jv\" (UID: \"109db1fc-2eed-4932-b461-7acbf67f88e7\") " pod="openshift-image-registry/node-ca-2k8jv" Sep 18 23:58:31.379132 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.378945 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-run-multus-certs\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.379132 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.378983 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-etc-kubernetes\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.379132 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379030 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-etc-kubernetes\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.379132 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379043 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-sysctl-d\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.379132 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379065 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9fe2a3fa-f8a3-4890-8bfc-2914fe2d336f-agent-certs\") pod \"konnectivity-agent-qsdhf\" (UID: \"9fe2a3fa-f8a3-4890-8bfc-2914fe2d336f\") " pod="kube-system/konnectivity-agent-qsdhf" Sep 18 23:58:31.379132 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379066 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-run-multus-certs\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.379132 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379115 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjvqt\" (UniqueName: \"kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt\") pod \"network-check-target-lpqlq\" (UID: \"1c03ca88-a515-4cf1-b753-c8e93034644b\") " pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379148 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-hostroot\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379174 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-slash\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379179 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-hostroot\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379198 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-lib-modules\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379212 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-slash\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379225 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6aa6323-b7b5-4f5b-8426-904ec24f1d79-host-slash\") pod \"iptables-alerter-r2q4h\" (UID: \"b6aa6323-b7b5-4f5b-8426-904ec24f1d79\") " pod="openshift-network-operator/iptables-alerter-r2q4h" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379255 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6691ea3d-128e-47ee-b52b-2a48ca799dc8-cnibin\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379280 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-etc-selinux\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379289 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b6aa6323-b7b5-4f5b-8426-904ec24f1d79-host-slash\") pod \"iptables-alerter-r2q4h\" (UID: \"b6aa6323-b7b5-4f5b-8426-904ec24f1d79\") " pod="openshift-network-operator/iptables-alerter-r2q4h" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379300 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6523199f-3f99-4e7f-ac81-4462f800b35d-cni-binary-copy\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379303 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfhqh\" (UniqueName: \"kubernetes.io/projected/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-kube-api-access-zfhqh\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379342 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6691ea3d-128e-47ee-b52b-2a48ca799dc8-cnibin\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379376 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-multus-cni-dir\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379400 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-var-lib-openvswitch\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379423 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-run-openvswitch\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379437 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-node-log\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.379509 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379438 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-var-lib-openvswitch\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379458 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ad0b3adb-7a95-473f-b3c9-2ac12f522495-ovn-node-metrics-cert\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379478 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-var-lib-kubelet\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379481 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-run-openvswitch\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379522 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9fe2a3fa-f8a3-4890-8bfc-2914fe2d336f-konnectivity-ca\") pod \"konnectivity-agent-qsdhf\" (UID: \"9fe2a3fa-f8a3-4890-8bfc-2914fe2d336f\") " pod="kube-system/konnectivity-agent-qsdhf" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379522 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-multus-cni-dir\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379559 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-cnibin\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379582 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-var-lib-cni-multus\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379560 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-node-log\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379609 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-etc-openvswitch\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379632 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-var-lib-cni-bin\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379653 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-var-lib-cni-multus\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379655 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-run-k8s-cni-cncf-io\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379680 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-run-k8s-cni-cncf-io\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379682 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ad0b3adb-7a95-473f-b3c9-2ac12f522495-ovnkube-config\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379712 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ad0b3adb-7a95-473f-b3c9-2ac12f522495-env-overrides\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379731 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6691ea3d-128e-47ee-b52b-2a48ca799dc8-cni-binary-copy\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.380289 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379747 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-registration-dir\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379778 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-run-ovn\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379825 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6691ea3d-128e-47ee-b52b-2a48ca799dc8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379849 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/109db1fc-2eed-4932-b461-7acbf67f88e7-host\") pod \"node-ca-2k8jv\" (UID: \"109db1fc-2eed-4932-b461-7acbf67f88e7\") " pod="openshift-image-registry/node-ca-2k8jv" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379871 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-cnibin\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379877 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-multus-conf-dir\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379902 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6523199f-3f99-4e7f-ac81-4462f800b35d-multus-daemon-config\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.379912 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-var-lib-cni-bin\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.380240 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-run-ovn\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.380255 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.380279 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blnjv\" (UniqueName: \"kubernetes.io/projected/b6aa6323-b7b5-4f5b-8426-904ec24f1d79-kube-api-access-blnjv\") pod \"iptables-alerter-r2q4h\" (UID: \"b6aa6323-b7b5-4f5b-8426-904ec24f1d79\") " pod="openshift-network-operator/iptables-alerter-r2q4h" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.380318 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c29d9e92-fea6-47b8-b299-ec7ae98f3010-dbus\") pod \"global-pull-secret-syncer-7nzpw\" (UID: \"c29d9e92-fea6-47b8-b299-ec7ae98f3010\") " pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.380344 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-cni-netd\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.380347 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ad0b3adb-7a95-473f-b3c9-2ac12f522495-env-overrides\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.380368 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-sysctl-conf\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.380395 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret\") pod \"global-pull-secret-syncer-7nzpw\" (UID: \"c29d9e92-fea6-47b8-b299-ec7ae98f3010\") " pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.380470 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ad0b3adb-7a95-473f-b3c9-2ac12f522495-ovnkube-config\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:31.380474 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 23:58:31.381165 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.380525 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c29d9e92-fea6-47b8-b299-ec7ae98f3010-dbus\") pod \"global-pull-secret-syncer-7nzpw\" (UID: \"c29d9e92-fea6-47b8-b299-ec7ae98f3010\") " pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:31.381990 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.380563 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-multus-conf-dir\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.381990 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:31.380596 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret podName:c29d9e92-fea6-47b8-b299-ec7ae98f3010 nodeName:}" failed. No retries permitted until 2026-09-18 23:58:31.880537845 +0000 UTC m=+3.034131709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret") pod "global-pull-secret-syncer-7nzpw" (UID: "c29d9e92-fea6-47b8-b299-ec7ae98f3010") : object "kube-system"/"original-pull-secret" not registered Sep 18 23:58:31.381990 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.380824 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6691ea3d-128e-47ee-b52b-2a48ca799dc8-cni-binary-copy\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.381990 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.380870 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-cni-netd\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.381990 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.380941 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.381990 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381044 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-etc-openvswitch\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.381990 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381106 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-device-dir\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.381990 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381130 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.381990 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381165 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6691ea3d-128e-47ee-b52b-2a48ca799dc8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.381990 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381198 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-system-cni-dir\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.381990 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381235 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh5kd\" (UniqueName: \"kubernetes.io/projected/6523199f-3f99-4e7f-ac81-4462f800b35d-kube-api-access-dh5kd\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.381990 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381258 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6523199f-3f99-4e7f-ac81-4462f800b35d-multus-daemon-config\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.381990 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381267 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c29d9e92-fea6-47b8-b299-ec7ae98f3010-kubelet-config\") pod \"global-pull-secret-syncer-7nzpw\" (UID: \"c29d9e92-fea6-47b8-b299-ec7ae98f3010\") " pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:31.381990 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381298 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-system-cni-dir\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.381990 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381346 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c29d9e92-fea6-47b8-b299-ec7ae98f3010-kubelet-config\") pod \"global-pull-secret-syncer-7nzpw\" (UID: \"c29d9e92-fea6-47b8-b299-ec7ae98f3010\") " pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:31.381990 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381354 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-log-socket\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381373 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-log-socket\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381394 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vmvg\" (UniqueName: \"kubernetes.io/projected/ad0b3adb-7a95-473f-b3c9-2ac12f522495-kube-api-access-4vmvg\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381421 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fljtw\" (UniqueName: \"kubernetes.io/projected/1df5be81-ad93-46f5-9886-90e729bdc767-kube-api-access-fljtw\") pod \"node-resolver-frlfs\" (UID: \"1df5be81-ad93-46f5-9886-90e729bdc767\") " pod="openshift-dns/node-resolver-frlfs" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381446 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-tuned\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381471 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-tmp\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381499 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-systemd-units\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381559 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-run-systemd\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381604 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-run-ovn-kubernetes\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381623 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-run-systemd\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381634 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6691ea3d-128e-47ee-b52b-2a48ca799dc8-system-cni-dir\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381659 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-systemd-units\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381702 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6691ea3d-128e-47ee-b52b-2a48ca799dc8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381738 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6691ea3d-128e-47ee-b52b-2a48ca799dc8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381772 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-run-ovn-kubernetes\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381768 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381850 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-modprobe-d\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.382757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381874 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-cni-bin\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381899 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-sysconfig\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381920 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-kubernetes\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381954 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6691ea3d-128e-47ee-b52b-2a48ca799dc8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.381741 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6691ea3d-128e-47ee-b52b-2a48ca799dc8-system-cni-dir\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382002 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-systemd\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382014 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-cni-bin\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382071 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-run\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382103 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-sys\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382139 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs\") pod \"network-metrics-daemon-g2zzv\" (UID: \"0092b756-14b9-4468-b25e-47a279c0e9a1\") " pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382332 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p69sr\" (UniqueName: \"kubernetes.io/projected/0092b756-14b9-4468-b25e-47a279c0e9a1-kube-api-access-p69sr\") pod \"network-metrics-daemon-g2zzv\" (UID: \"0092b756-14b9-4468-b25e-47a279c0e9a1\") " pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382374 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-var-lib-kubelet\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382395 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b6aa6323-b7b5-4f5b-8426-904ec24f1d79-iptables-alerter-script\") pod \"iptables-alerter-r2q4h\" (UID: \"b6aa6323-b7b5-4f5b-8426-904ec24f1d79\") " pod="openshift-network-operator/iptables-alerter-r2q4h" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382435 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ad0b3adb-7a95-473f-b3c9-2ac12f522495-ovnkube-script-lib\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382470 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1df5be81-ad93-46f5-9886-90e729bdc767-hosts-file\") pod \"node-resolver-frlfs\" (UID: \"1df5be81-ad93-46f5-9886-90e729bdc767\") " pod="openshift-dns/node-resolver-frlfs" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382510 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1df5be81-ad93-46f5-9886-90e729bdc767-tmp-dir\") pod \"node-resolver-frlfs\" (UID: \"1df5be81-ad93-46f5-9886-90e729bdc767\") " pod="openshift-dns/node-resolver-frlfs" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382536 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-host\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.383485 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382563 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-multus-socket-dir-parent\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.384270 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382587 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-kubelet\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.384270 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382611 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-host-var-lib-kubelet\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.384270 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382623 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfnqq\" (UniqueName: \"kubernetes.io/projected/6691ea3d-128e-47ee-b52b-2a48ca799dc8-kube-api-access-nfnqq\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.384270 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382669 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-sys-fs\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.384270 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382705 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2jrh\" (UniqueName: \"kubernetes.io/projected/109db1fc-2eed-4932-b461-7acbf67f88e7-kube-api-access-x2jrh\") pod \"node-ca-2k8jv\" (UID: \"109db1fc-2eed-4932-b461-7acbf67f88e7\") " pod="openshift-image-registry/node-ca-2k8jv" Sep 18 23:58:31.384270 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.382917 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6523199f-3f99-4e7f-ac81-4462f800b35d-multus-socket-dir-parent\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.384270 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.383027 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad0b3adb-7a95-473f-b3c9-2ac12f522495-host-kubelet\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.384270 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.383035 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1df5be81-ad93-46f5-9886-90e729bdc767-hosts-file\") pod \"node-resolver-frlfs\" (UID: \"1df5be81-ad93-46f5-9886-90e729bdc767\") " pod="openshift-dns/node-resolver-frlfs" Sep 18 23:58:31.384270 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.383713 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6691ea3d-128e-47ee-b52b-2a48ca799dc8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.384270 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.384175 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1df5be81-ad93-46f5-9886-90e729bdc767-tmp-dir\") pod \"node-resolver-frlfs\" (UID: \"1df5be81-ad93-46f5-9886-90e729bdc767\") " pod="openshift-dns/node-resolver-frlfs" Sep 18 23:58:31.384270 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.384182 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b6aa6323-b7b5-4f5b-8426-904ec24f1d79-iptables-alerter-script\") pod \"iptables-alerter-r2q4h\" (UID: \"b6aa6323-b7b5-4f5b-8426-904ec24f1d79\") " pod="openshift-network-operator/iptables-alerter-r2q4h" Sep 18 23:58:31.384270 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.384256 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ad0b3adb-7a95-473f-b3c9-2ac12f522495-ovnkube-script-lib\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.385036 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.385017 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ad0b3adb-7a95-473f-b3c9-2ac12f522495-ovn-node-metrics-cert\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.388832 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.388810 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blnjv\" (UniqueName: \"kubernetes.io/projected/b6aa6323-b7b5-4f5b-8426-904ec24f1d79-kube-api-access-blnjv\") pod \"iptables-alerter-r2q4h\" (UID: \"b6aa6323-b7b5-4f5b-8426-904ec24f1d79\") " pod="openshift-network-operator/iptables-alerter-r2q4h" Sep 18 23:58:31.390203 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.390182 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh5kd\" (UniqueName: \"kubernetes.io/projected/6523199f-3f99-4e7f-ac81-4462f800b35d-kube-api-access-dh5kd\") pod \"multus-wbld8\" (UID: \"6523199f-3f99-4e7f-ac81-4462f800b35d\") " pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.390477 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.390462 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fljtw\" (UniqueName: \"kubernetes.io/projected/1df5be81-ad93-46f5-9886-90e729bdc767-kube-api-access-fljtw\") pod \"node-resolver-frlfs\" (UID: \"1df5be81-ad93-46f5-9886-90e729bdc767\") " pod="openshift-dns/node-resolver-frlfs" Sep 18 23:58:31.391030 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.391011 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vmvg\" (UniqueName: \"kubernetes.io/projected/ad0b3adb-7a95-473f-b3c9-2ac12f522495-kube-api-access-4vmvg\") pod \"ovnkube-node-mmfk5\" (UID: \"ad0b3adb-7a95-473f-b3c9-2ac12f522495\") " pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.392098 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.392071 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfnqq\" (UniqueName: \"kubernetes.io/projected/6691ea3d-128e-47ee-b52b-2a48ca799dc8-kube-api-access-nfnqq\") pod \"multus-additional-cni-plugins-24d8b\" (UID: \"6691ea3d-128e-47ee-b52b-2a48ca799dc8\") " pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.427214 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.427164 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 23:58:31.483527 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483495 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-sys-fs\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.483698 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483529 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2jrh\" (UniqueName: \"kubernetes.io/projected/109db1fc-2eed-4932-b461-7acbf67f88e7-kube-api-access-x2jrh\") pod \"node-ca-2k8jv\" (UID: \"109db1fc-2eed-4932-b461-7acbf67f88e7\") " pod="openshift-image-registry/node-ca-2k8jv" Sep 18 23:58:31.483698 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483554 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-socket-dir\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.483698 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483641 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-sys-fs\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.483698 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483643 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq9b5\" (UniqueName: \"kubernetes.io/projected/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-kube-api-access-cq9b5\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.483698 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483695 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/109db1fc-2eed-4932-b461-7acbf67f88e7-serviceca\") pod \"node-ca-2k8jv\" (UID: \"109db1fc-2eed-4932-b461-7acbf67f88e7\") " pod="openshift-image-registry/node-ca-2k8jv" Sep 18 23:58:31.483988 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483699 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-socket-dir\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.483988 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483718 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-sysctl-d\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.483988 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483742 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9fe2a3fa-f8a3-4890-8bfc-2914fe2d336f-agent-certs\") pod \"konnectivity-agent-qsdhf\" (UID: \"9fe2a3fa-f8a3-4890-8bfc-2914fe2d336f\") " pod="kube-system/konnectivity-agent-qsdhf" Sep 18 23:58:31.483988 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483785 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjvqt\" (UniqueName: \"kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt\") pod \"network-check-target-lpqlq\" (UID: \"1c03ca88-a515-4cf1-b753-c8e93034644b\") " pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:31.483988 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483826 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-lib-modules\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.483988 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483850 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-etc-selinux\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.483988 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483875 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfhqh\" (UniqueName: \"kubernetes.io/projected/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-kube-api-access-zfhqh\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.483988 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483904 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-var-lib-kubelet\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.483988 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483919 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-sysctl-d\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.483988 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483930 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9fe2a3fa-f8a3-4890-8bfc-2914fe2d336f-konnectivity-ca\") pod \"konnectivity-agent-qsdhf\" (UID: \"9fe2a3fa-f8a3-4890-8bfc-2914fe2d336f\") " pod="kube-system/konnectivity-agent-qsdhf" Sep 18 23:58:31.483988 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.483974 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-registration-dir\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484000 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/109db1fc-2eed-4932-b461-7acbf67f88e7-host\") pod \"node-ca-2k8jv\" (UID: \"109db1fc-2eed-4932-b461-7acbf67f88e7\") " pod="openshift-image-registry/node-ca-2k8jv" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484028 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-sysctl-conf\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484066 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-device-dir\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484096 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-tuned\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484117 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-tmp\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484132 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/109db1fc-2eed-4932-b461-7acbf67f88e7-serviceca\") pod \"node-ca-2k8jv\" (UID: \"109db1fc-2eed-4932-b461-7acbf67f88e7\") " pod="openshift-image-registry/node-ca-2k8jv" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484145 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484168 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-modprobe-d\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484195 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-sysconfig\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484216 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-kubernetes\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484238 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-systemd\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484242 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-etc-selinux\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484262 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-run\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484281 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-sys\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484301 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs\") pod \"network-metrics-daemon-g2zzv\" (UID: \"0092b756-14b9-4468-b25e-47a279c0e9a1\") " pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484321 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-lib-modules\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484325 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p69sr\" (UniqueName: \"kubernetes.io/projected/0092b756-14b9-4468-b25e-47a279c0e9a1-kube-api-access-p69sr\") pod \"network-metrics-daemon-g2zzv\" (UID: \"0092b756-14b9-4468-b25e-47a279c0e9a1\") " pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:31.484522 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484353 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-host\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.485303 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484396 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-modprobe-d\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.485303 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484404 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9fe2a3fa-f8a3-4890-8bfc-2914fe2d336f-konnectivity-ca\") pod \"konnectivity-agent-qsdhf\" (UID: \"9fe2a3fa-f8a3-4890-8bfc-2914fe2d336f\") " pod="kube-system/konnectivity-agent-qsdhf" Sep 18 23:58:31.485303 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484430 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-host\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.485303 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484442 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/109db1fc-2eed-4932-b461-7acbf67f88e7-host\") pod \"node-ca-2k8jv\" (UID: \"109db1fc-2eed-4932-b461-7acbf67f88e7\") " pod="openshift-image-registry/node-ca-2k8jv" Sep 18 23:58:31.485303 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484461 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-kubernetes\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.485303 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484489 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-run\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.485303 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484520 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-registration-dir\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.485303 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484545 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-var-lib-kubelet\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.485303 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484577 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-systemd\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.485303 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:31.484588 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 23:58:31.485303 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484621 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-sysconfig\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.485303 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:31.484652 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs podName:0092b756-14b9-4468-b25e-47a279c0e9a1 nodeName:}" failed. No retries permitted until 2026-09-18 23:58:31.984635407 +0000 UTC m=+3.138229250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs") pod "network-metrics-daemon-g2zzv" (UID: "0092b756-14b9-4468-b25e-47a279c0e9a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 23:58:31.485303 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484835 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-device-dir\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.485303 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484871 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.485303 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484888 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-sysctl-conf\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.485303 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.484896 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-sys\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.486849 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.486828 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9fe2a3fa-f8a3-4890-8bfc-2914fe2d336f-agent-certs\") pod \"konnectivity-agent-qsdhf\" (UID: \"9fe2a3fa-f8a3-4890-8bfc-2914fe2d336f\") " pod="kube-system/konnectivity-agent-qsdhf" Sep 18 23:58:31.486996 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.486973 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-tmp\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.487104 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.487061 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-etc-tuned\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.490599 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:31.490577 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 23:58:31.490701 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:31.490606 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 23:58:31.490701 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:31.490620 2568 projected.go:196] Error preparing data for projected volume kube-api-access-zjvqt for pod openshift-network-diagnostics/network-check-target-lpqlq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 23:58:31.490701 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:31.490680 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt podName:1c03ca88-a515-4cf1-b753-c8e93034644b nodeName:}" failed. No retries permitted until 2026-09-18 23:58:31.990662957 +0000 UTC m=+3.144256823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zjvqt" (UniqueName: "kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt") pod "network-check-target-lpqlq" (UID: "1c03ca88-a515-4cf1-b753-c8e93034644b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 23:58:31.491979 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.491944 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2jrh\" (UniqueName: \"kubernetes.io/projected/109db1fc-2eed-4932-b461-7acbf67f88e7-kube-api-access-x2jrh\") pod \"node-ca-2k8jv\" (UID: \"109db1fc-2eed-4932-b461-7acbf67f88e7\") " pod="openshift-image-registry/node-ca-2k8jv" Sep 18 23:58:31.493014 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.492964 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq9b5\" (UniqueName: \"kubernetes.io/projected/cbdb4e96-0680-4cd1-acbe-449e2cbc8eba-kube-api-access-cq9b5\") pod \"tuned-9mgwp\" (UID: \"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba\") " pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.493229 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.493212 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfhqh\" (UniqueName: \"kubernetes.io/projected/4feb2ca4-d96a-4c9d-a210-212f2a9d077b-kube-api-access-zfhqh\") pod \"aws-ebs-csi-driver-node-pkvhg\" (UID: \"4feb2ca4-d96a-4c9d-a210-212f2a9d077b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.493343 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.493320 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p69sr\" (UniqueName: \"kubernetes.io/projected/0092b756-14b9-4468-b25e-47a279c0e9a1-kube-api-access-p69sr\") pod \"network-metrics-daemon-g2zzv\" (UID: \"0092b756-14b9-4468-b25e-47a279c0e9a1\") " pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:31.568676 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.568648 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:31.575724 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.575701 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-frlfs" Sep 18 23:58:31.587419 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.587401 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-24d8b" Sep 18 23:58:31.592969 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.592946 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wbld8" Sep 18 23:58:31.599414 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.599399 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r2q4h" Sep 18 23:58:31.606875 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.606856 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2k8jv" Sep 18 23:58:31.614465 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.614445 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qsdhf" Sep 18 23:58:31.622964 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.622946 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" Sep 18 23:58:31.628598 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.628583 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" Sep 18 23:58:31.773738 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.773655 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 23:58:31.888490 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.888459 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret\") pod \"global-pull-secret-syncer-7nzpw\" (UID: \"c29d9e92-fea6-47b8-b299-ec7ae98f3010\") " pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:31.888676 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:31.888619 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 23:58:31.888722 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:31.888688 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret podName:c29d9e92-fea6-47b8-b299-ec7ae98f3010 nodeName:}" failed. No retries permitted until 2026-09-18 23:58:32.888670044 +0000 UTC m=+4.042263887 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret") pod "global-pull-secret-syncer-7nzpw" (UID: "c29d9e92-fea6-47b8-b299-ec7ae98f3010") : object "kube-system"/"original-pull-secret" not registered Sep 18 23:58:31.989746 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:31.989716 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs\") pod \"network-metrics-daemon-g2zzv\" (UID: \"0092b756-14b9-4468-b25e-47a279c0e9a1\") " pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:31.989949 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:31.989841 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 23:58:31.989949 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:31.989898 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs podName:0092b756-14b9-4468-b25e-47a279c0e9a1 nodeName:}" failed. No retries permitted until 2026-09-18 23:58:32.989885814 +0000 UTC m=+4.143479658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs") pod "network-metrics-daemon-g2zzv" (UID: "0092b756-14b9-4468-b25e-47a279c0e9a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 23:58:32.090494 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.090404 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjvqt\" (UniqueName: \"kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt\") pod \"network-check-target-lpqlq\" (UID: \"1c03ca88-a515-4cf1-b753-c8e93034644b\") " pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:32.090636 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:32.090564 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 23:58:32.090636 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:32.090585 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 23:58:32.090636 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:32.090598 2568 projected.go:196] Error preparing data for projected volume kube-api-access-zjvqt for pod openshift-network-diagnostics/network-check-target-lpqlq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 23:58:32.090763 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:32.090658 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt podName:1c03ca88-a515-4cf1-b753-c8e93034644b nodeName:}" failed. No retries permitted until 2026-09-18 23:58:33.090640957 +0000 UTC m=+4.244234806 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zjvqt" (UniqueName: "kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt") pod "network-check-target-lpqlq" (UID: "1c03ca88-a515-4cf1-b753-c8e93034644b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 23:58:32.238888 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:32.238856 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6aa6323_b7b5_4f5b_8426_904ec24f1d79.slice/crio-b17c8db6e152d23aacfb490c7ec750e8b5f24bb0bc98ae83ba7b8b547ea2c02d WatchSource:0}: Error finding container b17c8db6e152d23aacfb490c7ec750e8b5f24bb0bc98ae83ba7b8b547ea2c02d: Status 404 returned error can't find the container with id b17c8db6e152d23aacfb490c7ec750e8b5f24bb0bc98ae83ba7b8b547ea2c02d Sep 18 23:58:32.240378 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:32.240353 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad0b3adb_7a95_473f_b3c9_2ac12f522495.slice/crio-6270c31a5c3f454786e1a345d2ac786068b1d7965a33c091a0327b2284c39e4c WatchSource:0}: Error finding container 6270c31a5c3f454786e1a345d2ac786068b1d7965a33c091a0327b2284c39e4c: Status 404 returned error can't find the container with id 6270c31a5c3f454786e1a345d2ac786068b1d7965a33c091a0327b2284c39e4c Sep 18 23:58:32.262108 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:32.262078 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod109db1fc_2eed_4932_b461_7acbf67f88e7.slice/crio-81540e57c1b5f3a57e983dbc94ed7842d91eae645a4fa99177bbe5e5ba059f12 WatchSource:0}: Error finding container 81540e57c1b5f3a57e983dbc94ed7842d91eae645a4fa99177bbe5e5ba059f12: Status 404 returned error can't find the container with id 81540e57c1b5f3a57e983dbc94ed7842d91eae645a4fa99177bbe5e5ba059f12 Sep 18 23:58:32.262894 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:32.262859 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fe2a3fa_f8a3_4890_8bfc_2914fe2d336f.slice/crio-8b75c80e48b2e0dbc842de07b1fb8922c69accbaeddb5d999cab244a09779a3b WatchSource:0}: Error finding container 8b75c80e48b2e0dbc842de07b1fb8922c69accbaeddb5d999cab244a09779a3b: Status 404 returned error can't find the container with id 8b75c80e48b2e0dbc842de07b1fb8922c69accbaeddb5d999cab244a09779a3b Sep 18 23:58:32.264269 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:32.263777 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbdb4e96_0680_4cd1_acbe_449e2cbc8eba.slice/crio-b572787efaecee7f67957fe8e3438e5d49f017a18ea45a048baa43086a1185c6 WatchSource:0}: Error finding container b572787efaecee7f67957fe8e3438e5d49f017a18ea45a048baa43086a1185c6: Status 404 returned error can't find the container with id b572787efaecee7f67957fe8e3438e5d49f017a18ea45a048baa43086a1185c6 Sep 18 23:58:32.264740 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:32.264701 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1df5be81_ad93_46f5_9886_90e729bdc767.slice/crio-21954282e25423e7cef2573c1eb5e67f5589fbe66fba3785d361e61edfe13a48 WatchSource:0}: Error finding container 21954282e25423e7cef2573c1eb5e67f5589fbe66fba3785d361e61edfe13a48: Status 404 returned error can't find the container with id 21954282e25423e7cef2573c1eb5e67f5589fbe66fba3785d361e61edfe13a48 Sep 18 23:58:32.266899 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:32.266878 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6523199f_3f99_4e7f_ac81_4462f800b35d.slice/crio-de67fcd1d4c2713e0d9d105fab9f53ef02ea78db669b70f24cf2a728e755f9a7 WatchSource:0}: Error finding container de67fcd1d4c2713e0d9d105fab9f53ef02ea78db669b70f24cf2a728e755f9a7: Status 404 returned error can't find the container with id de67fcd1d4c2713e0d9d105fab9f53ef02ea78db669b70f24cf2a728e755f9a7 Sep 18 23:58:32.268054 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:58:32.268037 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6691ea3d_128e_47ee_b52b_2a48ca799dc8.slice/crio-9debee757f64a2d41f4301afdbe32749a4ae05b64f172054a6dc57e618e0a6d8 WatchSource:0}: Error finding container 9debee757f64a2d41f4301afdbe32749a4ae05b64f172054a6dc57e618e0a6d8: Status 404 returned error can't find the container with id 9debee757f64a2d41f4301afdbe32749a4ae05b64f172054a6dc57e618e0a6d8 Sep 18 23:58:32.318667 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.318640 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 23:53:31 +0000 UTC" deadline="2028-04-05 20:50:44.07787661 +0000 UTC" Sep 18 23:58:32.318667 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.318665 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13556h52m11.759214096s" Sep 18 23:58:32.374608 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.374548 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:32.374727 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:32.374670 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:58:32.382542 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.382518 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2k8jv" event={"ID":"109db1fc-2eed-4932-b461-7acbf67f88e7","Type":"ContainerStarted","Data":"81540e57c1b5f3a57e983dbc94ed7842d91eae645a4fa99177bbe5e5ba059f12"} Sep 18 23:58:32.383624 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.383602 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" event={"ID":"ad0b3adb-7a95-473f-b3c9-2ac12f522495","Type":"ContainerStarted","Data":"6270c31a5c3f454786e1a345d2ac786068b1d7965a33c091a0327b2284c39e4c"} Sep 18 23:58:32.388127 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.388102 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-146.ec2.internal" event={"ID":"bae7cc99672b5dbd116a90c47b4ad4c3","Type":"ContainerStarted","Data":"2fb84b0c4ba7cab0eda35c3af86967fd385cfc972ca966b6f9375e326b8fcfd4"} Sep 18 23:58:32.390946 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.390920 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" event={"ID":"4feb2ca4-d96a-4c9d-a210-212f2a9d077b","Type":"ContainerStarted","Data":"202a67a5dc8ebf5233e176d8a99c46702962c983f5e95e16ce94d4d4b4f1df2a"} Sep 18 23:58:32.394566 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.394547 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-24d8b" event={"ID":"6691ea3d-128e-47ee-b52b-2a48ca799dc8","Type":"ContainerStarted","Data":"9debee757f64a2d41f4301afdbe32749a4ae05b64f172054a6dc57e618e0a6d8"} Sep 18 23:58:32.395445 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.395429 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" event={"ID":"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba","Type":"ContainerStarted","Data":"b572787efaecee7f67957fe8e3438e5d49f017a18ea45a048baa43086a1185c6"} Sep 18 23:58:32.396477 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.396460 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qsdhf" event={"ID":"9fe2a3fa-f8a3-4890-8bfc-2914fe2d336f","Type":"ContainerStarted","Data":"8b75c80e48b2e0dbc842de07b1fb8922c69accbaeddb5d999cab244a09779a3b"} Sep 18 23:58:32.399648 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.399623 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r2q4h" event={"ID":"b6aa6323-b7b5-4f5b-8426-904ec24f1d79","Type":"ContainerStarted","Data":"b17c8db6e152d23aacfb490c7ec750e8b5f24bb0bc98ae83ba7b8b547ea2c02d"} Sep 18 23:58:32.400921 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.400870 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-146.ec2.internal" podStartSLOduration=2.400855724 podStartE2EDuration="2.400855724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 23:58:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:58:32.40061988 +0000 UTC m=+3.554213745" watchObservedRunningTime="2026-09-18 23:58:32.400855724 +0000 UTC m=+3.554449591" Sep 18 23:58:32.403848 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.403821 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wbld8" event={"ID":"6523199f-3f99-4e7f-ac81-4462f800b35d","Type":"ContainerStarted","Data":"de67fcd1d4c2713e0d9d105fab9f53ef02ea78db669b70f24cf2a728e755f9a7"} Sep 18 23:58:32.404983 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.404961 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-frlfs" event={"ID":"1df5be81-ad93-46f5-9886-90e729bdc767","Type":"ContainerStarted","Data":"21954282e25423e7cef2573c1eb5e67f5589fbe66fba3785d361e61edfe13a48"} Sep 18 23:58:32.896710 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.896626 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret\") pod \"global-pull-secret-syncer-7nzpw\" (UID: \"c29d9e92-fea6-47b8-b299-ec7ae98f3010\") " pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:32.896930 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:32.896785 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 23:58:32.896930 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:32.896869 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret podName:c29d9e92-fea6-47b8-b299-ec7ae98f3010 nodeName:}" failed. No retries permitted until 2026-09-18 23:58:34.896850329 +0000 UTC m=+6.050444187 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret") pod "global-pull-secret-syncer-7nzpw" (UID: "c29d9e92-fea6-47b8-b299-ec7ae98f3010") : object "kube-system"/"original-pull-secret" not registered Sep 18 23:58:32.997434 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:32.997385 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs\") pod \"network-metrics-daemon-g2zzv\" (UID: \"0092b756-14b9-4468-b25e-47a279c0e9a1\") " pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:32.997608 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:32.997566 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 23:58:32.997670 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:32.997629 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs podName:0092b756-14b9-4468-b25e-47a279c0e9a1 nodeName:}" failed. No retries permitted until 2026-09-18 23:58:34.997609736 +0000 UTC m=+6.151203579 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs") pod "network-metrics-daemon-g2zzv" (UID: "0092b756-14b9-4468-b25e-47a279c0e9a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 23:58:33.099069 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:33.099006 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjvqt\" (UniqueName: \"kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt\") pod \"network-check-target-lpqlq\" (UID: \"1c03ca88-a515-4cf1-b753-c8e93034644b\") " pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:33.099294 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:33.099269 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 23:58:33.099367 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:33.099297 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 23:58:33.099367 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:33.099311 2568 projected.go:196] Error preparing data for projected volume kube-api-access-zjvqt for pod openshift-network-diagnostics/network-check-target-lpqlq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 23:58:33.099494 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:33.099418 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt podName:1c03ca88-a515-4cf1-b753-c8e93034644b nodeName:}" failed. No retries permitted until 2026-09-18 23:58:35.099356426 +0000 UTC m=+6.252950270 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zjvqt" (UniqueName: "kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt") pod "network-check-target-lpqlq" (UID: "1c03ca88-a515-4cf1-b753-c8e93034644b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 23:58:33.318913 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:33.318823 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 23:53:31 +0000 UTC" deadline="2028-03-01 08:21:19.066488228 +0000 UTC" Sep 18 23:58:33.318913 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:33.318865 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12704h22m45.747628291s" Sep 18 23:58:33.377104 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:33.376380 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:33.377104 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:33.376503 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpqlq" podUID="1c03ca88-a515-4cf1-b753-c8e93034644b" Sep 18 23:58:33.377104 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:33.376936 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:33.377104 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:33.377033 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2zzv" podUID="0092b756-14b9-4468-b25e-47a279c0e9a1" Sep 18 23:58:33.425755 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:33.425616 2568 generic.go:356] "Generic (PLEG): container finished" podID="03c6c038cb8cb442ed4c13464d8fdf16" containerID="2506dad4b27554b6a3b80ebe6143ba88553febc60bdff1a767ec951e2bf91fd7" exitCode=0 Sep 18 23:58:33.427492 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:33.425645 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal" event={"ID":"03c6c038cb8cb442ed4c13464d8fdf16","Type":"ContainerDied","Data":"2506dad4b27554b6a3b80ebe6143ba88553febc60bdff1a767ec951e2bf91fd7"} Sep 18 23:58:34.374300 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:34.374031 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:34.374748 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:34.374409 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:58:34.458852 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:34.458441 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal" event={"ID":"03c6c038cb8cb442ed4c13464d8fdf16","Type":"ContainerStarted","Data":"24564eab3924ce2f884463440697e03ed042efd852162859b927dd2eac61f4fe"} Sep 18 23:58:34.472698 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:34.472630 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-146.ec2.internal" podStartSLOduration=4.472585203 podStartE2EDuration="4.472585203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 23:58:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:58:34.471755773 +0000 UTC m=+5.625349639" watchObservedRunningTime="2026-09-18 23:58:34.472585203 +0000 UTC m=+5.626179070" Sep 18 23:58:34.914847 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:34.914709 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret\") pod \"global-pull-secret-syncer-7nzpw\" (UID: \"c29d9e92-fea6-47b8-b299-ec7ae98f3010\") " pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:34.915591 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:34.915142 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 23:58:34.915591 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:34.915230 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret podName:c29d9e92-fea6-47b8-b299-ec7ae98f3010 nodeName:}" failed. No retries permitted until 2026-09-18 23:58:38.915206774 +0000 UTC m=+10.068800637 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret") pod "global-pull-secret-syncer-7nzpw" (UID: "c29d9e92-fea6-47b8-b299-ec7ae98f3010") : object "kube-system"/"original-pull-secret" not registered Sep 18 23:58:35.015210 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:35.015073 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs\") pod \"network-metrics-daemon-g2zzv\" (UID: \"0092b756-14b9-4468-b25e-47a279c0e9a1\") " pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:35.015389 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:35.015231 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 23:58:35.015389 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:35.015295 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs podName:0092b756-14b9-4468-b25e-47a279c0e9a1 nodeName:}" failed. No retries permitted until 2026-09-18 23:58:39.015277012 +0000 UTC m=+10.168870858 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs") pod "network-metrics-daemon-g2zzv" (UID: "0092b756-14b9-4468-b25e-47a279c0e9a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 23:58:35.115462 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:35.115424 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjvqt\" (UniqueName: \"kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt\") pod \"network-check-target-lpqlq\" (UID: \"1c03ca88-a515-4cf1-b753-c8e93034644b\") " pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:35.115697 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:35.115655 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 23:58:35.115697 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:35.115683 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 23:58:35.115697 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:35.115697 2568 projected.go:196] Error preparing data for projected volume kube-api-access-zjvqt for pod openshift-network-diagnostics/network-check-target-lpqlq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 23:58:35.115956 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:35.115755 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt podName:1c03ca88-a515-4cf1-b753-c8e93034644b nodeName:}" failed. No retries permitted until 2026-09-18 23:58:39.115736927 +0000 UTC m=+10.269330773 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zjvqt" (UniqueName: "kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt") pod "network-check-target-lpqlq" (UID: "1c03ca88-a515-4cf1-b753-c8e93034644b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 23:58:35.374967 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:35.374897 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:35.375388 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:35.375011 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpqlq" podUID="1c03ca88-a515-4cf1-b753-c8e93034644b" Sep 18 23:58:35.375472 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:35.375451 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:35.375587 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:35.375567 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2zzv" podUID="0092b756-14b9-4468-b25e-47a279c0e9a1" Sep 18 23:58:36.374632 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:36.374135 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:36.374632 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:36.374273 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:58:37.374121 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:37.374093 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:37.374545 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:37.374220 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpqlq" podUID="1c03ca88-a515-4cf1-b753-c8e93034644b" Sep 18 23:58:37.376887 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:37.374052 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:37.377016 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:37.376995 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2zzv" podUID="0092b756-14b9-4468-b25e-47a279c0e9a1" Sep 18 23:58:38.374382 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:38.374343 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:38.375322 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:38.374525 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:58:38.950183 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:38.950151 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret\") pod \"global-pull-secret-syncer-7nzpw\" (UID: \"c29d9e92-fea6-47b8-b299-ec7ae98f3010\") " pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:38.950373 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:38.950337 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 23:58:38.950438 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:38.950397 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret podName:c29d9e92-fea6-47b8-b299-ec7ae98f3010 nodeName:}" failed. No retries permitted until 2026-09-18 23:58:46.950379988 +0000 UTC m=+18.103973838 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret") pod "global-pull-secret-syncer-7nzpw" (UID: "c29d9e92-fea6-47b8-b299-ec7ae98f3010") : object "kube-system"/"original-pull-secret" not registered Sep 18 23:58:39.051578 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:39.051543 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs\") pod \"network-metrics-daemon-g2zzv\" (UID: \"0092b756-14b9-4468-b25e-47a279c0e9a1\") " pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:39.051748 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:39.051696 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 23:58:39.051834 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:39.051757 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs podName:0092b756-14b9-4468-b25e-47a279c0e9a1 nodeName:}" failed. No retries permitted until 2026-09-18 23:58:47.051738395 +0000 UTC m=+18.205332251 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs") pod "network-metrics-daemon-g2zzv" (UID: "0092b756-14b9-4468-b25e-47a279c0e9a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 23:58:39.152380 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:39.152346 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjvqt\" (UniqueName: \"kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt\") pod \"network-check-target-lpqlq\" (UID: \"1c03ca88-a515-4cf1-b753-c8e93034644b\") " pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:39.152545 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:39.152533 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 23:58:39.152613 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:39.152557 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 23:58:39.152613 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:39.152570 2568 projected.go:196] Error preparing data for projected volume kube-api-access-zjvqt for pod openshift-network-diagnostics/network-check-target-lpqlq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 23:58:39.152725 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:39.152624 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt podName:1c03ca88-a515-4cf1-b753-c8e93034644b nodeName:}" failed. No retries permitted until 2026-09-18 23:58:47.152605289 +0000 UTC m=+18.306199149 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zjvqt" (UniqueName: "kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt") pod "network-check-target-lpqlq" (UID: "1c03ca88-a515-4cf1-b753-c8e93034644b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 23:58:39.375313 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:39.375229 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:39.375708 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:39.375345 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2zzv" podUID="0092b756-14b9-4468-b25e-47a279c0e9a1" Sep 18 23:58:39.375708 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:39.375404 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:39.375708 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:39.375481 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpqlq" podUID="1c03ca88-a515-4cf1-b753-c8e93034644b" Sep 18 23:58:40.373900 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:40.373856 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:40.374085 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:40.373996 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:58:41.374938 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:41.374902 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:41.375392 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:41.375025 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpqlq" podUID="1c03ca88-a515-4cf1-b753-c8e93034644b" Sep 18 23:58:41.375392 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:41.375063 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:41.375392 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:41.375148 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2zzv" podUID="0092b756-14b9-4468-b25e-47a279c0e9a1" Sep 18 23:58:42.374635 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:42.374590 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:42.374835 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:42.374721 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:58:43.374925 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:43.374879 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:43.375386 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:43.374889 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:43.375386 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:43.375019 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2zzv" podUID="0092b756-14b9-4468-b25e-47a279c0e9a1" Sep 18 23:58:43.375386 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:43.375086 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpqlq" podUID="1c03ca88-a515-4cf1-b753-c8e93034644b" Sep 18 23:58:44.374221 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:44.374181 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:44.374383 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:44.374302 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:58:45.374331 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:45.374302 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:45.374723 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:45.374413 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpqlq" podUID="1c03ca88-a515-4cf1-b753-c8e93034644b" Sep 18 23:58:45.374723 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:45.374470 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:45.374723 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:45.374609 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2zzv" podUID="0092b756-14b9-4468-b25e-47a279c0e9a1" Sep 18 23:58:46.374065 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:46.373973 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:46.374254 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:46.374166 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:58:47.012859 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:47.012823 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret\") pod \"global-pull-secret-syncer-7nzpw\" (UID: \"c29d9e92-fea6-47b8-b299-ec7ae98f3010\") " pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:47.013336 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:47.012986 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 23:58:47.013336 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:47.013065 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret podName:c29d9e92-fea6-47b8-b299-ec7ae98f3010 nodeName:}" failed. No retries permitted until 2026-09-18 23:59:03.013043762 +0000 UTC m=+34.166637609 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret") pod "global-pull-secret-syncer-7nzpw" (UID: "c29d9e92-fea6-47b8-b299-ec7ae98f3010") : object "kube-system"/"original-pull-secret" not registered Sep 18 23:58:47.113226 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:47.113190 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs\") pod \"network-metrics-daemon-g2zzv\" (UID: \"0092b756-14b9-4468-b25e-47a279c0e9a1\") " pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:47.113380 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:47.113343 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 23:58:47.113430 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:47.113419 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs podName:0092b756-14b9-4468-b25e-47a279c0e9a1 nodeName:}" failed. No retries permitted until 2026-09-18 23:59:03.113399771 +0000 UTC m=+34.266993632 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs") pod "network-metrics-daemon-g2zzv" (UID: "0092b756-14b9-4468-b25e-47a279c0e9a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 23:58:47.214276 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:47.214237 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjvqt\" (UniqueName: \"kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt\") pod \"network-check-target-lpqlq\" (UID: \"1c03ca88-a515-4cf1-b753-c8e93034644b\") " pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:47.214451 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:47.214414 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 23:58:47.214451 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:47.214440 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 23:58:47.214561 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:47.214453 2568 projected.go:196] Error preparing data for projected volume kube-api-access-zjvqt for pod openshift-network-diagnostics/network-check-target-lpqlq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 23:58:47.214561 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:47.214521 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt podName:1c03ca88-a515-4cf1-b753-c8e93034644b nodeName:}" failed. No retries permitted until 2026-09-18 23:59:03.214502158 +0000 UTC m=+34.368096018 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zjvqt" (UniqueName: "kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt") pod "network-check-target-lpqlq" (UID: "1c03ca88-a515-4cf1-b753-c8e93034644b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 23:58:47.374641 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:47.374566 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:47.374836 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:47.374701 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpqlq" podUID="1c03ca88-a515-4cf1-b753-c8e93034644b" Sep 18 23:58:47.374836 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:47.374754 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:47.374959 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:47.374895 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2zzv" podUID="0092b756-14b9-4468-b25e-47a279c0e9a1" Sep 18 23:58:48.374843 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:48.374813 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:48.375266 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:48.374929 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:58:49.375292 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:49.374912 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:49.375292 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:49.375004 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpqlq" podUID="1c03ca88-a515-4cf1-b753-c8e93034644b" Sep 18 23:58:49.375292 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:49.375073 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:49.375292 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:49.375196 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2zzv" podUID="0092b756-14b9-4468-b25e-47a279c0e9a1" Sep 18 23:58:50.374062 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:50.374026 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:50.374182 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:50.374156 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:58:50.495535 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:50.494108 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" event={"ID":"cbdb4e96-0680-4cd1-acbe-449e2cbc8eba","Type":"ContainerStarted","Data":"0b15554e59c1be255925a0e1c91f4a7c01065048152fe6fff79555967d93a85d"} Sep 18 23:58:50.497122 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:50.496760 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qsdhf" event={"ID":"9fe2a3fa-f8a3-4890-8bfc-2914fe2d336f","Type":"ContainerStarted","Data":"b07e0efef27dfd3a33cd9b10cfe8391bdb7ec19a871a708b2a7f1430fa9496bc"} Sep 18 23:58:50.499431 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:50.499349 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wbld8" event={"ID":"6523199f-3f99-4e7f-ac81-4462f800b35d","Type":"ContainerStarted","Data":"c9f3e7e0c12cf09cbd072c2a857e7735b394ecd94a2a299b059eb86c4d64baed"} Sep 18 23:58:50.500630 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:50.500609 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-frlfs" event={"ID":"1df5be81-ad93-46f5-9886-90e729bdc767","Type":"ContainerStarted","Data":"508c54324dc3241c41f6d55b55c14709ff5a656d950fadaefbeb7f18349db785"} Sep 18 23:58:50.504030 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:50.504002 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" event={"ID":"ad0b3adb-7a95-473f-b3c9-2ac12f522495","Type":"ContainerStarted","Data":"18fd1004ee0965835fb9b81056323d80627f9d45a9a75b486ba304484fa4b45d"} Sep 18 23:58:50.505737 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:50.505368 2568 generic.go:356] "Generic (PLEG): container finished" podID="6691ea3d-128e-47ee-b52b-2a48ca799dc8" containerID="b49280c1a176cec75df3274f3fb08b2d9b6047404b9e0b254c6843b5575428d0" exitCode=0 Sep 18 23:58:50.505737 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:50.505404 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-24d8b" event={"ID":"6691ea3d-128e-47ee-b52b-2a48ca799dc8","Type":"ContainerDied","Data":"b49280c1a176cec75df3274f3fb08b2d9b6047404b9e0b254c6843b5575428d0"} Sep 18 23:58:50.512829 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:50.512603 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9mgwp" podStartSLOduration=3.54915981 podStartE2EDuration="21.512589098s" totalImagesPullingTime="17.963429288s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 23:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:58:50.51235191 +0000 UTC m=+21.665945787" watchObservedRunningTime="2026-09-18 23:58:50.512589098 +0000 UTC m=+21.666182962" Sep 18 23:58:50.526678 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:50.526630 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-frlfs" podStartSLOduration=3.5873284659999998 podStartE2EDuration="21.526614716s" totalImagesPullingTime="17.93928625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 23:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:58:50.526479944 +0000 UTC m=+21.680073825" watchObservedRunningTime="2026-09-18 23:58:50.526614716 +0000 UTC m=+21.680208581" Sep 18 23:58:50.566227 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:50.566106 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wbld8" podStartSLOduration=3.488496723 podStartE2EDuration="21.566085889s" totalImagesPullingTime="18.077589166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 23:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:58:50.565360514 +0000 UTC m=+21.718954379" watchObservedRunningTime="2026-09-18 23:58:50.566085889 +0000 UTC m=+21.719679755" Sep 18 23:58:50.578834 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:50.578764 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2k8jv" podStartSLOduration=3.615310212 podStartE2EDuration="21.578749759s" totalImagesPullingTime="17.963439547s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 23:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:58:50.578560008 +0000 UTC m=+21.732153873" watchObservedRunningTime="2026-09-18 23:58:50.578749759 +0000 UTC m=+21.732343623" Sep 18 23:58:50.592177 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:50.592136 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qsdhf" podStartSLOduration=11.176399863 podStartE2EDuration="21.592120404s" totalImagesPullingTime="10.415720541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 23:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:58:50.59202538 +0000 UTC m=+21.745619246" watchObservedRunningTime="2026-09-18 23:58:50.592120404 +0000 UTC m=+21.745714279" Sep 18 23:58:51.329224 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:51.329183 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qsdhf" Sep 18 23:58:51.374254 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:51.374218 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:51.374439 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:51.374269 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:51.374439 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:51.374348 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpqlq" podUID="1c03ca88-a515-4cf1-b753-c8e93034644b" Sep 18 23:58:51.374559 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:51.374441 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2zzv" podUID="0092b756-14b9-4468-b25e-47a279c0e9a1" Sep 18 23:58:51.509970 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:51.509927 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" event={"ID":"4feb2ca4-d96a-4c9d-a210-212f2a9d077b","Type":"ContainerStarted","Data":"2947d67541d6266ce61aac885f823e0382dc9055cce89b60533ce07961fecf97"} Sep 18 23:58:51.511544 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:51.511511 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2k8jv" event={"ID":"109db1fc-2eed-4932-b461-7acbf67f88e7","Type":"ContainerStarted","Data":"df5c5daaa5128b04a42b786e3a5661c4d96157f490831f99fc5fe2ea15488bb6"} Sep 18 23:58:51.515258 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:51.515204 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" event={"ID":"ad0b3adb-7a95-473f-b3c9-2ac12f522495","Type":"ContainerStarted","Data":"90fe0396148358deefcf49641d76ebf56f911f0f1aa3fd2256b5bb22e340eb35"} Sep 18 23:58:51.515372 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:51.515280 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" event={"ID":"ad0b3adb-7a95-473f-b3c9-2ac12f522495","Type":"ContainerStarted","Data":"0385478089a32c08614641f1426dce82a741beb70ae53a6f1b6bd3ebaf9fab72"} Sep 18 23:58:51.515372 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:51.515330 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" event={"ID":"ad0b3adb-7a95-473f-b3c9-2ac12f522495","Type":"ContainerStarted","Data":"99b85371e0d2dbdbdfe3e64c587140a450014c32b2a2ffc01fdd546ecf910d18"} Sep 18 23:58:51.515456 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:51.515376 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" event={"ID":"ad0b3adb-7a95-473f-b3c9-2ac12f522495","Type":"ContainerStarted","Data":"e05135dc19394d38e299a3f8184e3d889a4ab82dc90e2794e4972159611ed6ad"} Sep 18 23:58:51.515456 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:51.515425 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" event={"ID":"ad0b3adb-7a95-473f-b3c9-2ac12f522495","Type":"ContainerStarted","Data":"c91d651ec8985fa2ee7bad4bb678198433b09409ae70f4332634d28066eda636"} Sep 18 23:58:51.975549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:51.975311 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 18 23:58:52.353610 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:52.353505 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T23:58:51.975544545Z","UUID":"6d7ff375-8845-462d-8b89-b1d1720d587c","Handler":null,"Name":"","Endpoint":""} Sep 18 23:58:52.355225 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:52.355202 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 18 23:58:52.355350 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:52.355236 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 18 23:58:52.373945 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:52.373916 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:52.374097 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:52.374044 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:58:52.521325 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:52.521285 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" event={"ID":"4feb2ca4-d96a-4c9d-a210-212f2a9d077b","Type":"ContainerStarted","Data":"3815a9ed54253afc3fb587b22eaa7856696ba5f9973d4e176c2d647367e909a9"} Sep 18 23:58:52.523931 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:52.523900 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r2q4h" event={"ID":"b6aa6323-b7b5-4f5b-8426-904ec24f1d79","Type":"ContainerStarted","Data":"743909d033adb4434b727e9f3723f354c84934106d127a022e245c7d1ee1ac14"} Sep 18 23:58:53.374070 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:53.373968 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:53.374243 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:53.374113 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpqlq" podUID="1c03ca88-a515-4cf1-b753-c8e93034644b" Sep 18 23:58:53.374243 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:53.374143 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:53.374353 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:53.374260 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2zzv" podUID="0092b756-14b9-4468-b25e-47a279c0e9a1" Sep 18 23:58:53.529943 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:53.529907 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" event={"ID":"ad0b3adb-7a95-473f-b3c9-2ac12f522495","Type":"ContainerStarted","Data":"6a6e4fd3e666e178a1a25e48b6bf574e94d96b66d3cf05d4bca1ad8d82fafeed"} Sep 18 23:58:53.532614 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:53.532582 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" event={"ID":"4feb2ca4-d96a-4c9d-a210-212f2a9d077b","Type":"ContainerStarted","Data":"61de77b0ce018784b74765bd87e2fe6a61f509dee758521f669f33088ccc86ec"} Sep 18 23:58:53.549914 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:53.549859 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r2q4h" podStartSLOduration=6.580743403 podStartE2EDuration="24.549844085s" totalImagesPullingTime="17.969100682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 23:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:58:52.547181277 +0000 UTC m=+23.700775140" watchObservedRunningTime="2026-09-18 23:58:53.549844085 +0000 UTC m=+24.703437951" Sep 18 23:58:53.550370 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:53.550341 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkvhg" podStartSLOduration=4.403655043 podStartE2EDuration="24.550331497s" totalImagesPullingTime="20.146676454s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 23:58:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:58:53.549581462 +0000 UTC m=+24.703175328" watchObservedRunningTime="2026-09-18 23:58:53.550331497 +0000 UTC m=+24.703925396" Sep 18 23:58:54.374593 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:54.374556 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:54.374775 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:54.374683 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:58:55.257979 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:55.257951 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qsdhf" Sep 18 23:58:55.258513 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:55.258494 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qsdhf" Sep 18 23:58:55.374766 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:55.374579 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:55.374958 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:55.374594 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:55.374958 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:55.374878 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpqlq" podUID="1c03ca88-a515-4cf1-b753-c8e93034644b" Sep 18 23:58:55.375048 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:55.374974 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2zzv" podUID="0092b756-14b9-4468-b25e-47a279c0e9a1" Sep 18 23:58:55.541755 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:55.541673 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" event={"ID":"ad0b3adb-7a95-473f-b3c9-2ac12f522495","Type":"ContainerStarted","Data":"bedf039c44f9805b9f4fd2f86b8f4e56cfe715fc5f9d309e6baccd30fed5bb53"} Sep 18 23:58:55.542033 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:55.542002 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:55.543656 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:55.543634 2568 generic.go:356] "Generic (PLEG): container finished" podID="6691ea3d-128e-47ee-b52b-2a48ca799dc8" containerID="7dc4ca59d3c42bfbee064d1e68abc255e1b6881d6a3117ec3f1f4ea8039a01f1" exitCode=0 Sep 18 23:58:55.543757 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:55.543715 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-24d8b" event={"ID":"6691ea3d-128e-47ee-b52b-2a48ca799dc8","Type":"ContainerDied","Data":"7dc4ca59d3c42bfbee064d1e68abc255e1b6881d6a3117ec3f1f4ea8039a01f1"} Sep 18 23:58:55.544459 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:55.544444 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qsdhf" Sep 18 23:58:55.558535 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:55.558513 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:55.565153 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:55.565106 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" podStartSLOduration=8.584864495 podStartE2EDuration="26.5650907s" totalImagesPullingTime="17.980226205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 23:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:58:55.563485151 +0000 UTC m=+26.717079015" watchObservedRunningTime="2026-09-18 23:58:55.5650907 +0000 UTC m=+26.718684569" Sep 18 23:58:56.374772 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:56.374732 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:56.375395 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:56.374872 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:58:56.546387 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:56.546293 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:56.546875 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:56.546781 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:56.564852 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:56.564816 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:58:57.303148 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:57.303104 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lpqlq"] Sep 18 23:58:57.303323 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:57.303256 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:57.303392 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:57.303372 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpqlq" podUID="1c03ca88-a515-4cf1-b753-c8e93034644b" Sep 18 23:58:57.303778 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:57.303756 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7nzpw"] Sep 18 23:58:57.303901 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:57.303863 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:57.303978 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:57.303959 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:58:57.305195 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:57.305033 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g2zzv"] Sep 18 23:58:57.305195 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:57.305140 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:57.305349 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:57.305234 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2zzv" podUID="0092b756-14b9-4468-b25e-47a279c0e9a1" Sep 18 23:58:57.550763 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:57.550727 2568 generic.go:356] "Generic (PLEG): container finished" podID="6691ea3d-128e-47ee-b52b-2a48ca799dc8" containerID="13c0578ac29cf4e59d317dd9590f0c57f4a71379f90501b5f085cf52e83af2b8" exitCode=0 Sep 18 23:58:57.551447 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:57.550822 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-24d8b" event={"ID":"6691ea3d-128e-47ee-b52b-2a48ca799dc8","Type":"ContainerDied","Data":"13c0578ac29cf4e59d317dd9590f0c57f4a71379f90501b5f085cf52e83af2b8"} Sep 18 23:58:59.375275 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:59.375101 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:58:59.375715 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:59.375160 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:58:59.375715 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:59.375180 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:58:59.375715 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:59.375354 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2zzv" podUID="0092b756-14b9-4468-b25e-47a279c0e9a1" Sep 18 23:58:59.375715 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:59.375409 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpqlq" podUID="1c03ca88-a515-4cf1-b753-c8e93034644b" Sep 18 23:58:59.375715 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:58:59.375487 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:58:59.557749 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:59.557677 2568 generic.go:356] "Generic (PLEG): container finished" podID="6691ea3d-128e-47ee-b52b-2a48ca799dc8" containerID="c7be27c733619c1c9de54e6b644b5fb640cf893d4199770a4f4688806b4bf972" exitCode=0 Sep 18 23:58:59.557749 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:58:59.557717 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-24d8b" event={"ID":"6691ea3d-128e-47ee-b52b-2a48ca799dc8","Type":"ContainerDied","Data":"c7be27c733619c1c9de54e6b644b5fb640cf893d4199770a4f4688806b4bf972"} Sep 18 23:59:01.375250 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:01.374918 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:59:01.375250 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:01.374944 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:59:01.375250 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:01.375063 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nzpw" podUID="c29d9e92-fea6-47b8-b299-ec7ae98f3010" Sep 18 23:59:01.375250 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:01.375137 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:59:01.375250 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:01.375142 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpqlq" podUID="1c03ca88-a515-4cf1-b753-c8e93034644b" Sep 18 23:59:01.375250 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:01.375216 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g2zzv" podUID="0092b756-14b9-4468-b25e-47a279c0e9a1" Sep 18 23:59:03.026561 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.026470 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret\") pod \"global-pull-secret-syncer-7nzpw\" (UID: \"c29d9e92-fea6-47b8-b299-ec7ae98f3010\") " pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:59:03.026966 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.026604 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 23:59:03.026966 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.026667 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret podName:c29d9e92-fea6-47b8-b299-ec7ae98f3010 nodeName:}" failed. No retries permitted until 2026-09-18 23:59:35.026653487 +0000 UTC m=+66.180247330 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret") pod "global-pull-secret-syncer-7nzpw" (UID: "c29d9e92-fea6-47b8-b299-ec7ae98f3010") : object "kube-system"/"original-pull-secret" not registered Sep 18 23:59:03.127623 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.127592 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs\") pod \"network-metrics-daemon-g2zzv\" (UID: \"0092b756-14b9-4468-b25e-47a279c0e9a1\") " pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:59:03.127773 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.127737 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 23:59:03.127865 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.127825 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs podName:0092b756-14b9-4468-b25e-47a279c0e9a1 nodeName:}" failed. No retries permitted until 2026-09-18 23:59:35.127788785 +0000 UTC m=+66.281382633 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs") pod "network-metrics-daemon-g2zzv" (UID: "0092b756-14b9-4468-b25e-47a279c0e9a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 23:59:03.142746 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.142721 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-146.ec2.internal" event="NodeReady" Sep 18 23:59:03.142896 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.142883 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 18 23:59:03.180147 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.180113 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76584b8696-td9lt"] Sep 18 23:59:03.183585 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.183560 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.186080 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.186058 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-q8qxt\"" Sep 18 23:59:03.186209 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.186108 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 18 23:59:03.186209 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.186069 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 18 23:59:03.186209 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.186137 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 18 23:59:03.196310 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.196289 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-j4srb"] Sep 18 23:59:03.196658 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.196634 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 18 23:59:03.199397 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.199378 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2zk5g"] Sep 18 23:59:03.199520 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.199505 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j4srb" Sep 18 23:59:03.201993 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.201966 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 18 23:59:03.202109 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.201968 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kfzh4\"" Sep 18 23:59:03.202249 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.202111 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 18 23:59:03.202787 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.202444 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76584b8696-td9lt"] Sep 18 23:59:03.202787 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.202450 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 18 23:59:03.202787 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.202521 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:03.205478 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.205455 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ljxt7\"" Sep 18 23:59:03.205686 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.205667 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 18 23:59:03.205835 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.205811 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 18 23:59:03.205964 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.205740 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j4srb"] Sep 18 23:59:03.206846 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.206721 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2zk5g"] Sep 18 23:59:03.214566 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.214545 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7s9nr"] Sep 18 23:59:03.217426 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.217408 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:03.220419 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.220397 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 18 23:59:03.220565 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.220544 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2jsqb\"" Sep 18 23:59:03.220682 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.220412 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 18 23:59:03.220784 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.220771 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 18 23:59:03.220933 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.220443 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 18 23:59:03.227938 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.227919 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7s9nr"] Sep 18 23:59:03.228142 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.228114 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjvqt\" (UniqueName: \"kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt\") pod \"network-check-target-lpqlq\" (UID: \"1c03ca88-a515-4cf1-b753-c8e93034644b\") " pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:59:03.228253 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.228238 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 23:59:03.228315 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.228261 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 23:59:03.228315 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.228273 2568 projected.go:196] Error preparing data for projected volume kube-api-access-zjvqt for pod openshift-network-diagnostics/network-check-target-lpqlq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 23:59:03.228411 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.228341 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt podName:1c03ca88-a515-4cf1-b753-c8e93034644b nodeName:}" failed. No retries permitted until 2026-09-18 23:59:35.22832263 +0000 UTC m=+66.381916479 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zjvqt" (UniqueName: "kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt") pod "network-check-target-lpqlq" (UID: "1c03ca88-a515-4cf1-b753-c8e93034644b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 23:59:03.329078 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329005 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.329078 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329036 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-installation-pull-secrets\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.329078 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329057 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1db5e3f-6ee9-4059-bde8-7c257b1f343d-cert\") pod \"ingress-canary-j4srb\" (UID: \"f1db5e3f-6ee9-4059-bde8-7c257b1f343d\") " pod="openshift-ingress-canary/ingress-canary-j4srb" Sep 18 23:59:03.329078 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329079 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-config-volume\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:03.329078 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329102 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4wjl\" (UniqueName: \"kubernetes.io/projected/f1db5e3f-6ee9-4059-bde8-7c257b1f343d-kube-api-access-p4wjl\") pod \"ingress-canary-j4srb\" (UID: \"f1db5e3f-6ee9-4059-bde8-7c257b1f343d\") " pod="openshift-ingress-canary/ingress-canary-j4srb" Sep 18 23:59:03.329474 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329178 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-ca-trust-extracted\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.329474 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329279 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:03.329474 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329319 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-image-registry-private-configuration\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.329474 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329360 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-bound-sa-token\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.329474 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329404 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv82w\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-kube-api-access-jv82w\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.329474 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329440 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-certificates\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.329474 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329457 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-tmp-dir\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:03.329474 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329471 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ed90be5c-28d1-4406-982a-e165bf2211e1-crio-socket\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:03.329830 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329490 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdwgt\" (UniqueName: \"kubernetes.io/projected/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-api-access-hdwgt\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:03.329830 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329533 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:03.329830 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329555 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqj6m\" (UniqueName: \"kubernetes.io/projected/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-kube-api-access-nqj6m\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:03.329830 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329573 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-trusted-ca\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.329830 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329597 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed90be5c-28d1-4406-982a-e165bf2211e1-data-volume\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:03.329830 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.329643 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:03.374627 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.374592 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:59:03.374820 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.374601 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:59:03.374820 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.374729 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:59:03.377377 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.377073 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 18 23:59:03.377377 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.377206 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jgg6h\"" Sep 18 23:59:03.377377 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.377222 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 18 23:59:03.377377 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.377260 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 18 23:59:03.377377 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.377210 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ggrtp\"" Sep 18 23:59:03.377712 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.377467 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 18 23:59:03.430077 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430033 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.430201 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430081 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-installation-pull-secrets\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.430201 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430108 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1db5e3f-6ee9-4059-bde8-7c257b1f343d-cert\") pod \"ingress-canary-j4srb\" (UID: \"f1db5e3f-6ee9-4059-bde8-7c257b1f343d\") " pod="openshift-ingress-canary/ingress-canary-j4srb" Sep 18 23:59:03.430201 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430141 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-config-volume\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:03.430201 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430167 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4wjl\" (UniqueName: \"kubernetes.io/projected/f1db5e3f-6ee9-4059-bde8-7c257b1f343d-kube-api-access-p4wjl\") pod \"ingress-canary-j4srb\" (UID: \"f1db5e3f-6ee9-4059-bde8-7c257b1f343d\") " pod="openshift-ingress-canary/ingress-canary-j4srb" Sep 18 23:59:03.430406 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430203 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-ca-trust-extracted\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.430406 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.430208 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 23:59:03.430406 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.430228 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76584b8696-td9lt: secret "image-registry-tls" not found Sep 18 23:59:03.430406 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430262 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:03.430406 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.430290 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls podName:85f8145e-e418-47cc-9dc7-f8501c9a8e0f nodeName:}" failed. No retries permitted until 2026-09-18 23:59:03.930268972 +0000 UTC m=+35.083862820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls") pod "image-registry-76584b8696-td9lt" (UID: "85f8145e-e418-47cc-9dc7-f8501c9a8e0f") : secret "image-registry-tls" not found Sep 18 23:59:03.430406 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430331 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-image-registry-private-configuration\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.430406 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430363 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-bound-sa-token\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.430406 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430391 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv82w\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-kube-api-access-jv82w\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.430832 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430420 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-certificates\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.430832 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430440 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-tmp-dir\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:03.430832 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430456 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ed90be5c-28d1-4406-982a-e165bf2211e1-crio-socket\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:03.430832 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430477 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdwgt\" (UniqueName: \"kubernetes.io/projected/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-api-access-hdwgt\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:03.430832 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.430536 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 23:59:03.430832 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.430592 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-rbac-proxy-cm podName:ed90be5c-28d1-4406-982a-e165bf2211e1 nodeName:}" failed. No retries permitted until 2026-09-18 23:59:03.930576045 +0000 UTC m=+35.084169890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7s9nr" (UID: "ed90be5c-28d1-4406-982a-e165bf2211e1") : configmap "kube-rbac-proxy" not found Sep 18 23:59:03.430832 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430608 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-ca-trust-extracted\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.430832 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430616 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:03.430832 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430643 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqj6m\" (UniqueName: \"kubernetes.io/projected/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-kube-api-access-nqj6m\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:03.430832 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430670 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-trusted-ca\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.430832 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430693 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed90be5c-28d1-4406-982a-e165bf2211e1-data-volume\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:03.430832 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430720 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:03.431441 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.430856 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 23:59:03.431441 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430890 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-config-volume\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:03.431441 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.430897 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls podName:ed90be5c-28d1-4406-982a-e165bf2211e1 nodeName:}" failed. No retries permitted until 2026-09-18 23:59:03.930883709 +0000 UTC m=+35.084477555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7s9nr" (UID: "ed90be5c-28d1-4406-982a-e165bf2211e1") : secret "insights-runtime-extractor-tls" not found Sep 18 23:59:03.431441 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.430934 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-tmp-dir\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:03.431441 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.431007 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 23:59:03.431441 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.431046 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls podName:d8619f0e-fc65-4fbf-9dd2-2c612a94997f nodeName:}" failed. No retries permitted until 2026-09-18 23:59:03.931033946 +0000 UTC m=+35.084627788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls") pod "dns-default-2zk5g" (UID: "d8619f0e-fc65-4fbf-9dd2-2c612a94997f") : secret "dns-default-metrics-tls" not found Sep 18 23:59:03.431441 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.431105 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ed90be5c-28d1-4406-982a-e165bf2211e1-crio-socket\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:03.431441 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.431116 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-certificates\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.431441 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.431286 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed90be5c-28d1-4406-982a-e165bf2211e1-data-volume\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:03.431945 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.431783 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-trusted-ca\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.435354 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.435329 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-image-registry-private-configuration\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.435354 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.435349 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1db5e3f-6ee9-4059-bde8-7c257b1f343d-cert\") pod \"ingress-canary-j4srb\" (UID: \"f1db5e3f-6ee9-4059-bde8-7c257b1f343d\") " pod="openshift-ingress-canary/ingress-canary-j4srb" Sep 18 23:59:03.435513 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.435339 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-installation-pull-secrets\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.442164 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.442137 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdwgt\" (UniqueName: \"kubernetes.io/projected/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-api-access-hdwgt\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:03.442266 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.442136 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4wjl\" (UniqueName: \"kubernetes.io/projected/f1db5e3f-6ee9-4059-bde8-7c257b1f343d-kube-api-access-p4wjl\") pod \"ingress-canary-j4srb\" (UID: \"f1db5e3f-6ee9-4059-bde8-7c257b1f343d\") " pod="openshift-ingress-canary/ingress-canary-j4srb" Sep 18 23:59:03.442266 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.442246 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-bound-sa-token\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.442715 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.442691 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv82w\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-kube-api-access-jv82w\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.443098 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.443081 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqj6m\" (UniqueName: \"kubernetes.io/projected/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-kube-api-access-nqj6m\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:03.511788 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.511751 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j4srb" Sep 18 23:59:03.746306 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.746152 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-frlfs_1df5be81-ad93-46f5-9886-90e729bdc767/dns-node-resolver/0.log" Sep 18 23:59:03.934319 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.934282 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:03.934460 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.934341 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:03.934460 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.934373 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:03.934460 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.934402 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 23:59:03.934460 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.934412 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:03.934641 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.934476 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-rbac-proxy-cm podName:ed90be5c-28d1-4406-982a-e165bf2211e1 nodeName:}" failed. No retries permitted until 2026-09-18 23:59:04.934454507 +0000 UTC m=+36.088048363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7s9nr" (UID: "ed90be5c-28d1-4406-982a-e165bf2211e1") : configmap "kube-rbac-proxy" not found Sep 18 23:59:03.934641 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.934498 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 23:59:03.934641 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.934506 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 23:59:03.934641 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.934522 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76584b8696-td9lt: secret "image-registry-tls" not found Sep 18 23:59:03.934641 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.934497 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 23:59:03.934641 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.934538 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls podName:ed90be5c-28d1-4406-982a-e165bf2211e1 nodeName:}" failed. No retries permitted until 2026-09-18 23:59:04.934524636 +0000 UTC m=+36.088118481 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7s9nr" (UID: "ed90be5c-28d1-4406-982a-e165bf2211e1") : secret "insights-runtime-extractor-tls" not found Sep 18 23:59:03.934641 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.934565 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls podName:d8619f0e-fc65-4fbf-9dd2-2c612a94997f nodeName:}" failed. No retries permitted until 2026-09-18 23:59:04.934553429 +0000 UTC m=+36.088147272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls") pod "dns-default-2zk5g" (UID: "d8619f0e-fc65-4fbf-9dd2-2c612a94997f") : secret "dns-default-metrics-tls" not found Sep 18 23:59:03.934641 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:03.934578 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls podName:85f8145e-e418-47cc-9dc7-f8501c9a8e0f nodeName:}" failed. No retries permitted until 2026-09-18 23:59:04.934571599 +0000 UTC m=+36.088165441 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls") pod "image-registry-76584b8696-td9lt" (UID: "85f8145e-e418-47cc-9dc7-f8501c9a8e0f") : secret "image-registry-tls" not found Sep 18 23:59:03.943139 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.943115 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-flxrv"] Sep 18 23:59:03.946421 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.946402 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-flxrv" Sep 18 23:59:03.948675 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.948654 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 18 23:59:03.948831 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.948695 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-g5dr4\"" Sep 18 23:59:03.948972 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.948959 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 18 23:59:03.953116 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.953096 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-flxrv"] Sep 18 23:59:03.975983 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:03.975914 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j4srb"] Sep 18 23:59:03.982853 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:59:03.982826 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1db5e3f_6ee9_4059_bde8_7c257b1f343d.slice/crio-5339428db7237e925aa79491d43ae3a7c9e3c9cb051a065d7be3a0fb9e4a11ab WatchSource:0}: Error finding container 5339428db7237e925aa79491d43ae3a7c9e3c9cb051a065d7be3a0fb9e4a11ab: Status 404 returned error can't find the container with id 5339428db7237e925aa79491d43ae3a7c9e3c9cb051a065d7be3a0fb9e4a11ab Sep 18 23:59:04.035445 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:04.035418 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4jlq\" (UniqueName: \"kubernetes.io/projected/41c42309-992e-4bec-a4bc-9ce6de529b4e-kube-api-access-s4jlq\") pod \"migrator-76794cbd6c-flxrv\" (UID: \"41c42309-992e-4bec-a4bc-9ce6de529b4e\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-flxrv" Sep 18 23:59:04.136082 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:04.136053 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4jlq\" (UniqueName: \"kubernetes.io/projected/41c42309-992e-4bec-a4bc-9ce6de529b4e-kube-api-access-s4jlq\") pod \"migrator-76794cbd6c-flxrv\" (UID: \"41c42309-992e-4bec-a4bc-9ce6de529b4e\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-flxrv" Sep 18 23:59:04.143679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:04.143660 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4jlq\" (UniqueName: \"kubernetes.io/projected/41c42309-992e-4bec-a4bc-9ce6de529b4e-kube-api-access-s4jlq\") pod \"migrator-76794cbd6c-flxrv\" (UID: \"41c42309-992e-4bec-a4bc-9ce6de529b4e\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-flxrv" Sep 18 23:59:04.290115 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:04.290027 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-flxrv" Sep 18 23:59:04.557945 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:04.557865 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-flxrv"] Sep 18 23:59:04.561638 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:59:04.561614 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41c42309_992e_4bec_a4bc_9ce6de529b4e.slice/crio-b4f9417ef3eef60370e5a7de0013a156fca519b0b6d3be1c411bbe0224bf9a67 WatchSource:0}: Error finding container b4f9417ef3eef60370e5a7de0013a156fca519b0b6d3be1c411bbe0224bf9a67: Status 404 returned error can't find the container with id b4f9417ef3eef60370e5a7de0013a156fca519b0b6d3be1c411bbe0224bf9a67 Sep 18 23:59:04.571780 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:04.571713 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-flxrv" event={"ID":"41c42309-992e-4bec-a4bc-9ce6de529b4e","Type":"ContainerStarted","Data":"b4f9417ef3eef60370e5a7de0013a156fca519b0b6d3be1c411bbe0224bf9a67"} Sep 18 23:59:04.572927 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:04.572894 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j4srb" event={"ID":"f1db5e3f-6ee9-4059-bde8-7c257b1f343d","Type":"ContainerStarted","Data":"5339428db7237e925aa79491d43ae3a7c9e3c9cb051a065d7be3a0fb9e4a11ab"} Sep 18 23:59:04.576097 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:04.576069 2568 generic.go:356] "Generic (PLEG): container finished" podID="6691ea3d-128e-47ee-b52b-2a48ca799dc8" containerID="1919e903b0c508044d72cd428ccfae3fb65d646c58aae1088fcadd8efbb769dc" exitCode=0 Sep 18 23:59:04.576199 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:04.576121 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-24d8b" event={"ID":"6691ea3d-128e-47ee-b52b-2a48ca799dc8","Type":"ContainerDied","Data":"1919e903b0c508044d72cd428ccfae3fb65d646c58aae1088fcadd8efbb769dc"} Sep 18 23:59:04.730489 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:04.730460 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2k8jv_109db1fc-2eed-4932-b461-7acbf67f88e7/node-ca/0.log" Sep 18 23:59:04.944355 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:04.944325 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:04.944534 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:04.944384 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:04.944534 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:04.944418 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:04.944534 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:04.944452 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:04.944534 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:04.944465 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 23:59:04.944725 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:04.944533 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 23:59:04.944725 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:04.944539 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-rbac-proxy-cm podName:ed90be5c-28d1-4406-982a-e165bf2211e1 nodeName:}" failed. No retries permitted until 2026-09-18 23:59:06.9445192 +0000 UTC m=+38.098113049 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7s9nr" (UID: "ed90be5c-28d1-4406-982a-e165bf2211e1") : configmap "kube-rbac-proxy" not found Sep 18 23:59:04.944725 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:04.944571 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 23:59:04.944725 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:04.944573 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 23:59:04.944725 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:04.944588 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76584b8696-td9lt: secret "image-registry-tls" not found Sep 18 23:59:04.944725 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:04.944579 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls podName:d8619f0e-fc65-4fbf-9dd2-2c612a94997f nodeName:}" failed. No retries permitted until 2026-09-18 23:59:06.944569497 +0000 UTC m=+38.098163340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls") pod "dns-default-2zk5g" (UID: "d8619f0e-fc65-4fbf-9dd2-2c612a94997f") : secret "dns-default-metrics-tls" not found Sep 18 23:59:04.944725 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:04.944654 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls podName:85f8145e-e418-47cc-9dc7-f8501c9a8e0f nodeName:}" failed. No retries permitted until 2026-09-18 23:59:06.944636988 +0000 UTC m=+38.098230845 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls") pod "image-registry-76584b8696-td9lt" (UID: "85f8145e-e418-47cc-9dc7-f8501c9a8e0f") : secret "image-registry-tls" not found Sep 18 23:59:04.944725 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:04.944669 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls podName:ed90be5c-28d1-4406-982a-e165bf2211e1 nodeName:}" failed. No retries permitted until 2026-09-18 23:59:06.944660178 +0000 UTC m=+38.098254020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7s9nr" (UID: "ed90be5c-28d1-4406-982a-e165bf2211e1") : secret "insights-runtime-extractor-tls" not found Sep 18 23:59:05.582918 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:05.582882 2568 generic.go:356] "Generic (PLEG): container finished" podID="6691ea3d-128e-47ee-b52b-2a48ca799dc8" containerID="938be1c620a3b9d1594da1d91382ce11b1e43059f29ddb05635281f2e99fac48" exitCode=0 Sep 18 23:59:05.583505 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:05.582935 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-24d8b" event={"ID":"6691ea3d-128e-47ee-b52b-2a48ca799dc8","Type":"ContainerDied","Data":"938be1c620a3b9d1594da1d91382ce11b1e43059f29ddb05635281f2e99fac48"} Sep 18 23:59:06.961181 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:06.960964 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:06.961181 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:06.961170 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:06.961698 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:06.961196 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:06.961698 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:06.961092 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 18 23:59:06.961698 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:06.961222 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:06.961698 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:06.961284 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-rbac-proxy-cm podName:ed90be5c-28d1-4406-982a-e165bf2211e1 nodeName:}" failed. No retries permitted until 2026-09-18 23:59:10.961261029 +0000 UTC m=+42.114854875 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7s9nr" (UID: "ed90be5c-28d1-4406-982a-e165bf2211e1") : configmap "kube-rbac-proxy" not found Sep 18 23:59:06.961698 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:06.961301 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 23:59:06.961698 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:06.961330 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 23:59:06.961698 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:06.961340 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76584b8696-td9lt: secret "image-registry-tls" not found Sep 18 23:59:06.961698 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:06.961350 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls podName:d8619f0e-fc65-4fbf-9dd2-2c612a94997f nodeName:}" failed. No retries permitted until 2026-09-18 23:59:10.961336285 +0000 UTC m=+42.114930128 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls") pod "dns-default-2zk5g" (UID: "d8619f0e-fc65-4fbf-9dd2-2c612a94997f") : secret "dns-default-metrics-tls" not found Sep 18 23:59:06.961698 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:06.961302 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 23:59:06.961698 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:06.961373 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls podName:85f8145e-e418-47cc-9dc7-f8501c9a8e0f nodeName:}" failed. No retries permitted until 2026-09-18 23:59:10.961362379 +0000 UTC m=+42.114956221 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls") pod "image-registry-76584b8696-td9lt" (UID: "85f8145e-e418-47cc-9dc7-f8501c9a8e0f") : secret "image-registry-tls" not found Sep 18 23:59:06.961698 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:06.961385 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls podName:ed90be5c-28d1-4406-982a-e165bf2211e1 nodeName:}" failed. No retries permitted until 2026-09-18 23:59:10.961379012 +0000 UTC m=+42.114972854 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7s9nr" (UID: "ed90be5c-28d1-4406-982a-e165bf2211e1") : secret "insights-runtime-extractor-tls" not found Sep 18 23:59:07.592082 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:07.592044 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-24d8b" event={"ID":"6691ea3d-128e-47ee-b52b-2a48ca799dc8","Type":"ContainerStarted","Data":"fabdfe914c3f478e8980902b3e30b849d08d76a5be535f32325ccfac696f9863"} Sep 18 23:59:07.593713 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:07.593682 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-flxrv" event={"ID":"41c42309-992e-4bec-a4bc-9ce6de529b4e","Type":"ContainerStarted","Data":"2610dbef692a12d031f06f082a5174bfe5545e610134e7631b9e087e971a7f78"} Sep 18 23:59:07.593861 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:07.593733 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-flxrv" event={"ID":"41c42309-992e-4bec-a4bc-9ce6de529b4e","Type":"ContainerStarted","Data":"464eeed7a3297c2de00d1de1ec2011a0bb0cfb3a24ba8aa2b2f23b417c463e01"} Sep 18 23:59:07.594992 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:07.594972 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j4srb" event={"ID":"f1db5e3f-6ee9-4059-bde8-7c257b1f343d","Type":"ContainerStarted","Data":"72331df5e01906db27fd975525497432a25a9a4a80f6d859133e377885a991f7"} Sep 18 23:59:07.616963 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:07.616911 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-24d8b" podStartSLOduration=14.111798349 podStartE2EDuration="38.616898949s" totalImagesPullingTime="24.5051006s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 23:58:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:59:07.615851649 +0000 UTC m=+38.769445517" watchObservedRunningTime="2026-09-18 23:59:07.616898949 +0000 UTC m=+38.770492813" Sep 18 23:59:07.633457 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:07.633409 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-j4srb" podStartSLOduration=2.034186782 podStartE2EDuration="4.633393929s" totalImagesPullingTime="2.599207147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 23:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:59:07.632783148 +0000 UTC m=+38.786377013" watchObservedRunningTime="2026-09-18 23:59:07.633393929 +0000 UTC m=+38.786987794" Sep 18 23:59:07.648723 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:07.648673 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-flxrv" podStartSLOduration=2.626498112 podStartE2EDuration="4.648658765s" totalImagesPullingTime="2.022160653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 23:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:59:07.64841633 +0000 UTC m=+38.802010204" watchObservedRunningTime="2026-09-18 23:59:07.648658765 +0000 UTC m=+38.802252632" Sep 18 23:59:10.993006 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:10.992967 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:10.993006 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:10.993007 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:10.993414 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:10.993034 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:10.993414 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:10.993136 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 18 23:59:10.993414 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:10.993151 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 18 23:59:10.993414 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:10.993161 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76584b8696-td9lt: secret "image-registry-tls" not found Sep 18 23:59:10.993414 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:10.993136 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 18 23:59:10.993414 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:10.993214 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls podName:ed90be5c-28d1-4406-982a-e165bf2211e1 nodeName:}" failed. No retries permitted until 2026-09-18 23:59:18.993193166 +0000 UTC m=+50.146787024 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7s9nr" (UID: "ed90be5c-28d1-4406-982a-e165bf2211e1") : secret "insights-runtime-extractor-tls" not found Sep 18 23:59:10.993414 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:10.993232 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls podName:85f8145e-e418-47cc-9dc7-f8501c9a8e0f nodeName:}" failed. No retries permitted until 2026-09-18 23:59:18.993222462 +0000 UTC m=+50.146816311 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls") pod "image-registry-76584b8696-td9lt" (UID: "85f8145e-e418-47cc-9dc7-f8501c9a8e0f") : secret "image-registry-tls" not found Sep 18 23:59:10.993414 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:10.993255 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls podName:d8619f0e-fc65-4fbf-9dd2-2c612a94997f nodeName:}" failed. No retries permitted until 2026-09-18 23:59:18.993236935 +0000 UTC m=+50.146830778 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls") pod "dns-default-2zk5g" (UID: "d8619f0e-fc65-4fbf-9dd2-2c612a94997f") : secret "dns-default-metrics-tls" not found Sep 18 23:59:10.993414 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:10.993193 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:10.993753 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:10.993734 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ed90be5c-28d1-4406-982a-e165bf2211e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:19.052028 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.051988 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:19.052028 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.052035 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:19.052525 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.052217 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:19.055475 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.055448 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d8619f0e-fc65-4fbf-9dd2-2c612a94997f-metrics-tls\") pod \"dns-default-2zk5g\" (UID: \"d8619f0e-fc65-4fbf-9dd2-2c612a94997f\") " pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:19.055591 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.055482 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls\") pod \"image-registry-76584b8696-td9lt\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:19.055591 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.055449 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ed90be5c-28d1-4406-982a-e165bf2211e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7s9nr\" (UID: \"ed90be5c-28d1-4406-982a-e165bf2211e1\") " pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:19.100948 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.100921 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:19.118784 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.118760 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:19.126953 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.126911 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7s9nr" Sep 18 23:59:19.389042 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.388933 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76584b8696-td9lt"] Sep 18 23:59:19.391621 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:59:19.391592 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85f8145e_e418_47cc_9dc7_f8501c9a8e0f.slice/crio-2f239e12044b1d1d3fb1b6dc7ba883d266a49ee1a70e9ac147c83fd6f1d9527e WatchSource:0}: Error finding container 2f239e12044b1d1d3fb1b6dc7ba883d266a49ee1a70e9ac147c83fd6f1d9527e: Status 404 returned error can't find the container with id 2f239e12044b1d1d3fb1b6dc7ba883d266a49ee1a70e9ac147c83fd6f1d9527e Sep 18 23:59:19.412510 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.412488 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2zk5g"] Sep 18 23:59:19.416016 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:59:19.415992 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8619f0e_fc65_4fbf_9dd2_2c612a94997f.slice/crio-da3690b07b345518c222ea32a9a5cd07ab97bd5a0be8ee823b9f78fa3c06e561 WatchSource:0}: Error finding container da3690b07b345518c222ea32a9a5cd07ab97bd5a0be8ee823b9f78fa3c06e561: Status 404 returned error can't find the container with id da3690b07b345518c222ea32a9a5cd07ab97bd5a0be8ee823b9f78fa3c06e561 Sep 18 23:59:19.435237 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.435211 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7s9nr"] Sep 18 23:59:19.446865 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:59:19.446840 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded90be5c_28d1_4406_982a_e165bf2211e1.slice/crio-ec460d72a7520a3f49cdb73063b6e84ccbe457313cb831a6584c55c8cf394cdf WatchSource:0}: Error finding container ec460d72a7520a3f49cdb73063b6e84ccbe457313cb831a6584c55c8cf394cdf: Status 404 returned error can't find the container with id ec460d72a7520a3f49cdb73063b6e84ccbe457313cb831a6584c55c8cf394cdf Sep 18 23:59:19.625349 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.625309 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76584b8696-td9lt" event={"ID":"85f8145e-e418-47cc-9dc7-f8501c9a8e0f","Type":"ContainerStarted","Data":"18c5f41522a3f6ddc05b69ac00a0e06ef13248a1c398c219072851d39dd5a8ca"} Sep 18 23:59:19.625533 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.625361 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76584b8696-td9lt" event={"ID":"85f8145e-e418-47cc-9dc7-f8501c9a8e0f","Type":"ContainerStarted","Data":"2f239e12044b1d1d3fb1b6dc7ba883d266a49ee1a70e9ac147c83fd6f1d9527e"} Sep 18 23:59:19.625533 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.625405 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:19.626621 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.626597 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2zk5g" event={"ID":"d8619f0e-fc65-4fbf-9dd2-2c612a94997f","Type":"ContainerStarted","Data":"da3690b07b345518c222ea32a9a5cd07ab97bd5a0be8ee823b9f78fa3c06e561"} Sep 18 23:59:19.628025 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.628005 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7s9nr" event={"ID":"ed90be5c-28d1-4406-982a-e165bf2211e1","Type":"ContainerStarted","Data":"caee71749307adaa6e64f5a33db4daa77d8ce1e51495a1ce7783cb9d728a3b74"} Sep 18 23:59:19.628112 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.628044 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7s9nr" event={"ID":"ed90be5c-28d1-4406-982a-e165bf2211e1","Type":"ContainerStarted","Data":"ec460d72a7520a3f49cdb73063b6e84ccbe457313cb831a6584c55c8cf394cdf"} Sep 18 23:59:19.643580 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:19.643497 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76584b8696-td9lt" podStartSLOduration=49.643479436 podStartE2EDuration="49.643479436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 23:58:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:59:19.643183451 +0000 UTC m=+50.796777333" watchObservedRunningTime="2026-09-18 23:59:19.643479436 +0000 UTC m=+50.797073301" Sep 18 23:59:20.638020 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:20.637970 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7s9nr" event={"ID":"ed90be5c-28d1-4406-982a-e165bf2211e1","Type":"ContainerStarted","Data":"28a945384632dd21a1c83d8d71142b5ccc4178d559dbae9acf97877447634a8f"} Sep 18 23:59:21.644214 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:21.644173 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2zk5g" event={"ID":"d8619f0e-fc65-4fbf-9dd2-2c612a94997f","Type":"ContainerStarted","Data":"66e6ff52790393ca7b0fce26675aeb4e597678473744131acccaa5b052bc4147"} Sep 18 23:59:21.644655 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:21.644242 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2zk5g" event={"ID":"d8619f0e-fc65-4fbf-9dd2-2c612a94997f","Type":"ContainerStarted","Data":"ec96be963d673246779ea1cd697a1f183bf73792e1657973c68f961cc8f9e43d"} Sep 18 23:59:21.644655 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:21.644366 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:21.660031 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:21.659975 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2zk5g" podStartSLOduration=16.829468272 podStartE2EDuration="18.659957264s" totalImagesPullingTime="1.830488992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 23:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:59:21.659003887 +0000 UTC m=+52.812597777" watchObservedRunningTime="2026-09-18 23:59:21.659957264 +0000 UTC m=+52.813551129" Sep 18 23:59:22.650059 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:22.650007 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7s9nr" event={"ID":"ed90be5c-28d1-4406-982a-e165bf2211e1","Type":"ContainerStarted","Data":"376856b17d4b716783a525a3d0b15b7455f45d9bc47a816cb47b84b1473c3093"} Sep 18 23:59:22.666899 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:22.666852 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7s9nr" podStartSLOduration=17.247374164 podStartE2EDuration="19.666837082s" totalImagesPullingTime="2.419462918s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 23:59:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:59:22.666425051 +0000 UTC m=+53.820018916" watchObservedRunningTime="2026-09-18 23:59:22.666837082 +0000 UTC m=+53.820430947" Sep 18 23:59:25.587996 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.587961 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt"] Sep 18 23:59:25.593041 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.593021 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.595357 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.595330 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 18 23:59:25.595551 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.595533 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 18 23:59:25.595651 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.595560 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 18 23:59:25.595651 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.595580 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 18 23:59:25.595651 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.595643 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 18 23:59:25.596409 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.596388 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 18 23:59:25.596495 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.596389 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 18 23:59:25.601487 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.601467 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt"] Sep 18 23:59:25.671155 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.671127 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76584b8696-td9lt"] Sep 18 23:59:25.707055 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.707018 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/57b69d42-4568-4c42-b670-cf41a4c48f39-ca\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.707188 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.707064 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/57b69d42-4568-4c42-b670-cf41a4c48f39-hub\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.707188 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.707138 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/57b69d42-4568-4c42-b670-cf41a4c48f39-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.707265 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.707189 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/57b69d42-4568-4c42-b670-cf41a4c48f39-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.707265 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.707222 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/57b69d42-4568-4c42-b670-cf41a4c48f39-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.707265 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.707243 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctj2t\" (UniqueName: \"kubernetes.io/projected/57b69d42-4568-4c42-b670-cf41a4c48f39-kube-api-access-ctj2t\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.807987 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.807952 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/57b69d42-4568-4c42-b670-cf41a4c48f39-ca\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.808182 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.808001 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/57b69d42-4568-4c42-b670-cf41a4c48f39-hub\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.808182 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.808040 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/57b69d42-4568-4c42-b670-cf41a4c48f39-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.808182 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.808066 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/57b69d42-4568-4c42-b670-cf41a4c48f39-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.808182 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.808101 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/57b69d42-4568-4c42-b670-cf41a4c48f39-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.808384 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.808318 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctj2t\" (UniqueName: \"kubernetes.io/projected/57b69d42-4568-4c42-b670-cf41a4c48f39-kube-api-access-ctj2t\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.808885 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.808863 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/57b69d42-4568-4c42-b670-cf41a4c48f39-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.810680 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.810653 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/57b69d42-4568-4c42-b670-cf41a4c48f39-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.810788 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.810753 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/57b69d42-4568-4c42-b670-cf41a4c48f39-hub\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.811029 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.811013 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/57b69d42-4568-4c42-b670-cf41a4c48f39-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.811100 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.811081 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/57b69d42-4568-4c42-b670-cf41a4c48f39-ca\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.818302 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.818283 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctj2t\" (UniqueName: \"kubernetes.io/projected/57b69d42-4568-4c42-b670-cf41a4c48f39-kube-api-access-ctj2t\") pod \"cluster-proxy-proxy-agent-64756cf4c8-mc9kt\" (UID: \"57b69d42-4568-4c42-b670-cf41a4c48f39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:25.912109 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:25.912083 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" Sep 18 23:59:26.179989 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:26.179913 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt"] Sep 18 23:59:26.182847 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:59:26.182823 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57b69d42_4568_4c42_b670_cf41a4c48f39.slice/crio-29917b445fa3ac89960bd0d0fd3744f3483fb00b8b9a60071eaa6bdad18ca44f WatchSource:0}: Error finding container 29917b445fa3ac89960bd0d0fd3744f3483fb00b8b9a60071eaa6bdad18ca44f: Status 404 returned error can't find the container with id 29917b445fa3ac89960bd0d0fd3744f3483fb00b8b9a60071eaa6bdad18ca44f Sep 18 23:59:26.663973 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:26.663941 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" event={"ID":"57b69d42-4568-4c42-b670-cf41a4c48f39","Type":"ContainerStarted","Data":"29917b445fa3ac89960bd0d0fd3744f3483fb00b8b9a60071eaa6bdad18ca44f"} Sep 18 23:59:28.570248 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:28.570214 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mmfk5" Sep 18 23:59:30.680916 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:30.680876 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" event={"ID":"57b69d42-4568-4c42-b670-cf41a4c48f39","Type":"ContainerStarted","Data":"70b0a8518fd1789098858837efefa1b0a211c0a9a8d6b7999fcfa93533899196"} Sep 18 23:59:30.680916 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:30.680925 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" event={"ID":"57b69d42-4568-4c42-b670-cf41a4c48f39","Type":"ContainerStarted","Data":"9092930842a61ba36a6c4e9660f44e6ec58ee057e2966c251dcdcaba06db7b57"} Sep 18 23:59:30.681396 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:30.680937 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" event={"ID":"57b69d42-4568-4c42-b670-cf41a4c48f39","Type":"ContainerStarted","Data":"a93638be8164bdc15c43721fa4a1076d92b64a28ed0761ecd5eb5164ce9889e6"} Sep 18 23:59:30.698241 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:30.698191 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64756cf4c8-mc9kt" podStartSLOduration=1.754231219 podStartE2EDuration="5.698176471s" totalImagesPullingTime="3.943945252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 23:59:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:59:30.696928969 +0000 UTC m=+61.850522833" watchObservedRunningTime="2026-09-18 23:59:30.698176471 +0000 UTC m=+61.851770336" Sep 18 23:59:31.652635 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:31.652606 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2zk5g" Sep 18 23:59:35.079744 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.079697 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret\") pod \"global-pull-secret-syncer-7nzpw\" (UID: \"c29d9e92-fea6-47b8-b299-ec7ae98f3010\") " pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:59:35.082083 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.082058 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 18 23:59:35.092307 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.092283 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c29d9e92-fea6-47b8-b299-ec7ae98f3010-original-pull-secret\") pod \"global-pull-secret-syncer-7nzpw\" (UID: \"c29d9e92-fea6-47b8-b299-ec7ae98f3010\") " pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:59:35.180054 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.180018 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs\") pod \"network-metrics-daemon-g2zzv\" (UID: \"0092b756-14b9-4468-b25e-47a279c0e9a1\") " pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:59:35.182433 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.182415 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 18 23:59:35.192734 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.192705 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0092b756-14b9-4468-b25e-47a279c0e9a1-metrics-certs\") pod \"network-metrics-daemon-g2zzv\" (UID: \"0092b756-14b9-4468-b25e-47a279c0e9a1\") " pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:59:35.197689 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.197670 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nzpw" Sep 18 23:59:35.286499 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.281136 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjvqt\" (UniqueName: \"kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt\") pod \"network-check-target-lpqlq\" (UID: \"1c03ca88-a515-4cf1-b753-c8e93034644b\") " pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:59:35.290311 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.290046 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 18 23:59:35.302369 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.302124 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 18 23:59:35.314679 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.314633 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjvqt\" (UniqueName: \"kubernetes.io/projected/1c03ca88-a515-4cf1-b753-c8e93034644b-kube-api-access-zjvqt\") pod \"network-check-target-lpqlq\" (UID: \"1c03ca88-a515-4cf1-b753-c8e93034644b\") " pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:59:35.487750 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.487726 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jgg6h\"" Sep 18 23:59:35.492459 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.492429 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7nzpw"] Sep 18 23:59:35.494379 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.494361 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ggrtp\"" Sep 18 23:59:35.495615 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.495596 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:59:35.495843 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:59:35.495824 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc29d9e92_fea6_47b8_b299_ec7ae98f3010.slice/crio-1d99d570cbfb8299a761582a9ecf8d350ad8a272933ba1d3d3ac4288592c12af WatchSource:0}: Error finding container 1d99d570cbfb8299a761582a9ecf8d350ad8a272933ba1d3d3ac4288592c12af: Status 404 returned error can't find the container with id 1d99d570cbfb8299a761582a9ecf8d350ad8a272933ba1d3d3ac4288592c12af Sep 18 23:59:35.503072 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.503056 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g2zzv" Sep 18 23:59:35.678058 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.678023 2568 patch_prober.go:28] interesting pod/image-registry-76584b8696-td9lt container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 18 23:59:35.678227 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.678083 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-76584b8696-td9lt" podUID="85f8145e-e418-47cc-9dc7-f8501c9a8e0f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 18 23:59:35.699398 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.699364 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7nzpw" event={"ID":"c29d9e92-fea6-47b8-b299-ec7ae98f3010","Type":"ContainerStarted","Data":"1d99d570cbfb8299a761582a9ecf8d350ad8a272933ba1d3d3ac4288592c12af"} Sep 18 23:59:35.795219 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.795159 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lpqlq"] Sep 18 23:59:35.797585 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:59:35.797560 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c03ca88_a515_4cf1_b753_c8e93034644b.slice/crio-b3ad727faff2ead4c0d7ae888f8d39cf8a72636773bd6b44ec947024d1781efb WatchSource:0}: Error finding container b3ad727faff2ead4c0d7ae888f8d39cf8a72636773bd6b44ec947024d1781efb: Status 404 returned error can't find the container with id b3ad727faff2ead4c0d7ae888f8d39cf8a72636773bd6b44ec947024d1781efb Sep 18 23:59:35.816400 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:35.816375 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g2zzv"] Sep 18 23:59:35.819171 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:59:35.819149 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0092b756_14b9_4468_b25e_47a279c0e9a1.slice/crio-2f110b48f998651ede78eb7b3ec5e4f1f91ef7c7e75c7ec8ab49c0324b83313a WatchSource:0}: Error finding container 2f110b48f998651ede78eb7b3ec5e4f1f91ef7c7e75c7ec8ab49c0324b83313a: Status 404 returned error can't find the container with id 2f110b48f998651ede78eb7b3ec5e4f1f91ef7c7e75c7ec8ab49c0324b83313a Sep 18 23:59:36.705010 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:36.704973 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lpqlq" event={"ID":"1c03ca88-a515-4cf1-b753-c8e93034644b","Type":"ContainerStarted","Data":"b3ad727faff2ead4c0d7ae888f8d39cf8a72636773bd6b44ec947024d1781efb"} Sep 18 23:59:36.706616 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:36.706577 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g2zzv" event={"ID":"0092b756-14b9-4468-b25e-47a279c0e9a1","Type":"ContainerStarted","Data":"2f110b48f998651ede78eb7b3ec5e4f1f91ef7c7e75c7ec8ab49c0324b83313a"} Sep 18 23:59:37.713741 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:37.713706 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g2zzv" event={"ID":"0092b756-14b9-4468-b25e-47a279c0e9a1","Type":"ContainerStarted","Data":"d712d13b0abb7f544c3a8003b3ba8b124c72f1d3edeb9421be6cc230704925e4"} Sep 18 23:59:37.714227 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:37.713788 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g2zzv" event={"ID":"0092b756-14b9-4468-b25e-47a279c0e9a1","Type":"ContainerStarted","Data":"1d8adf74e87fd699c83ea3d6da562d7e15d5d3513f021a110fdee19fffdfa4b6"} Sep 18 23:59:37.729029 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:37.728949 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-g2zzv" podStartSLOduration=67.21114427 podStartE2EDuration="1m8.728933941s" totalImagesPullingTime="1.517789671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 23:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:59:37.728211175 +0000 UTC m=+68.881805040" watchObservedRunningTime="2026-09-18 23:59:37.728933941 +0000 UTC m=+68.882527805" Sep 18 23:59:39.926625 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:39.926594 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ssw9v"] Sep 18 23:59:39.928941 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:39.928917 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:39.931354 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:39.931324 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 18 23:59:39.931697 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:39.931679 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 18 23:59:39.931824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:39.931699 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 18 23:59:39.931824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:39.931680 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 18 23:59:39.931824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:39.931680 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-v4xfs\"" Sep 18 23:59:39.932692 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:39.932419 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 18 23:59:39.932692 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:39.932421 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 18 23:59:40.015645 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.015612 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-root\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.015856 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.015658 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-wtmp\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.015856 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.015696 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-sys\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.015856 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.015714 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-metrics-client-ca\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.015856 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.015823 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.015856 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.015848 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-accelerators-collector-config\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.016078 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.015865 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-textfile\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.016078 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.015884 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-tls\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.016078 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.015980 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqnqm\" (UniqueName: \"kubernetes.io/projected/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-kube-api-access-lqnqm\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.117325 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.117288 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-textfile\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.117325 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.117329 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-tls\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.117545 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.117359 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqnqm\" (UniqueName: \"kubernetes.io/projected/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-kube-api-access-lqnqm\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.117545 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.117379 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-root\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.117545 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.117413 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-wtmp\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.117545 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.117461 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-sys\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.117545 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.117484 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-metrics-client-ca\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.117545 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:40.117488 2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 18 23:59:40.117545 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.117513 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.117545 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.117538 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-accelerators-collector-config\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.117950 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:40.117565 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-tls podName:e0f865cd-5066-4d6e-b11d-aa9ef1dd8702 nodeName:}" failed. No retries permitted until 2026-09-18 23:59:40.617544343 +0000 UTC m=+71.771138201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-tls") pod "node-exporter-ssw9v" (UID: "e0f865cd-5066-4d6e-b11d-aa9ef1dd8702") : secret "node-exporter-tls" not found Sep 18 23:59:40.117950 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.117660 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-textfile\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.117950 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.117830 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-wtmp\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.118117 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.118094 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-accelerators-collector-config\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.118178 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.118121 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-root\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.118178 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.118173 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-sys\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.118373 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.118354 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-metrics-client-ca\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.120683 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.120659 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.138323 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.138299 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqnqm\" (UniqueName: \"kubernetes.io/projected/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-kube-api-access-lqnqm\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.620414 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.620375 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-tls\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.623046 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.623011 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e0f865cd-5066-4d6e-b11d-aa9ef1dd8702-node-exporter-tls\") pod \"node-exporter-ssw9v\" (UID: \"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702\") " pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.726179 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.726082 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lpqlq" event={"ID":"1c03ca88-a515-4cf1-b753-c8e93034644b","Type":"ContainerStarted","Data":"be2b215a1ebdb7f7f03ab2a60d452a169b156dfe83a55638f85a604a428a546b"} Sep 18 23:59:40.726179 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.726172 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 18 23:59:40.727611 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.727588 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7nzpw" event={"ID":"c29d9e92-fea6-47b8-b299-ec7ae98f3010","Type":"ContainerStarted","Data":"f9f92c19b39a45f34a985d49f2a06c8a2f9426822152da3771a7876f6c05a2af"} Sep 18 23:59:40.745866 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.745817 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lpqlq" podStartSLOduration=67.100489428 podStartE2EDuration="1m11.745779424s" totalImagesPullingTime="4.645289996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 23:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:59:40.744161564 +0000 UTC m=+71.897755442" watchObservedRunningTime="2026-09-18 23:59:40.745779424 +0000 UTC m=+71.899373290" Sep 18 23:59:40.763009 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.762968 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7nzpw" podStartSLOduration=66.812849728 podStartE2EDuration="1m11.762954897s" totalImagesPullingTime="4.950105169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 23:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:59:40.762536424 +0000 UTC m=+71.916130289" watchObservedRunningTime="2026-09-18 23:59:40.762954897 +0000 UTC m=+71.916548762" Sep 18 23:59:40.840863 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.840830 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ssw9v" Sep 18 23:59:40.850543 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:59:40.850514 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0f865cd_5066_4d6e_b11d_aa9ef1dd8702.slice/crio-b9c208ab2e95cccd4be6142a7f4d78608388ca7a7fce292d510940fa3c0f5e54 WatchSource:0}: Error finding container b9c208ab2e95cccd4be6142a7f4d78608388ca7a7fce292d510940fa3c0f5e54: Status 404 returned error can't find the container with id b9c208ab2e95cccd4be6142a7f4d78608388ca7a7fce292d510940fa3c0f5e54 Sep 18 23:59:40.964461 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.964411 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 18 23:59:40.967420 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.967399 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:40.969753 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.969730 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 18 23:59:40.969902 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.969738 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 18 23:59:40.969902 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.969740 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 18 23:59:40.969902 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.969885 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 18 23:59:40.970120 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.970104 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 18 23:59:40.970120 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.970111 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 18 23:59:40.970462 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.970441 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jrj22\"" Sep 18 23:59:40.970563 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.970441 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 18 23:59:40.970563 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.970487 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 18 23:59:40.970689 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.970673 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 18 23:59:40.982423 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:40.982377 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 18 23:59:41.024275 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.024248 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-config-out\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.024275 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.024275 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sptx7\" (UniqueName: \"kubernetes.io/projected/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-kube-api-access-sptx7\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.024446 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.024295 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.024446 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.024316 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.024446 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.024332 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.024446 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.024390 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.024446 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.024427 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-web-config\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.024446 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.024446 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.024680 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.024462 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.024680 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.024489 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.024680 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.024521 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.024680 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.024543 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-config-volume\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.024680 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.024558 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.125641 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.125608 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.125824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.125654 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-config-volume\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.125824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.125674 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.125824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.125700 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-config-out\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.125824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.125721 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sptx7\" (UniqueName: \"kubernetes.io/projected/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-kube-api-access-sptx7\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.125824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.125752 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.125824 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.125786 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.126135 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.125837 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.126135 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:41.125857 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-alertmanager-trusted-ca-bundle podName:2d22a728-bbb0-4ea8-b27a-91823e1d8baa nodeName:}" failed. No retries permitted until 2026-09-18 23:59:41.625832645 +0000 UTC m=+72.779426511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "2d22a728-bbb0-4ea8-b27a-91823e1d8baa") : configmap references non-existent config key: ca-bundle.crt Sep 18 23:59:41.126135 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.125899 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.126135 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.125937 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-web-config\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.126135 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.125964 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.126135 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.125991 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.126135 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.126023 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.126836 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.126691 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.128286 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.128261 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.129462 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.129394 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.129462 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.129406 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-config-volume\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.129612 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:41.129502 2568 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 18 23:59:41.129612 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.129525 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.129612 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:41.129577 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-main-tls podName:2d22a728-bbb0-4ea8-b27a-91823e1d8baa nodeName:}" failed. No retries permitted until 2026-09-18 23:59:41.629552582 +0000 UTC m=+72.783146440 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "2d22a728-bbb0-4ea8-b27a-91823e1d8baa") : secret "alertmanager-main-tls" not found Sep 18 23:59:41.129747 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.129646 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-web-config\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.129747 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.129725 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.129840 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.129788 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.130106 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.130082 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-config-out\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.131364 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.131342 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.136751 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.136731 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sptx7\" (UniqueName: \"kubernetes.io/projected/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-kube-api-access-sptx7\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.629890 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.629858 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.630168 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.629944 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.630987 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.630959 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.632728 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.632696 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:41.732015 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.731977 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ssw9v" event={"ID":"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702","Type":"ContainerStarted","Data":"b9c208ab2e95cccd4be6142a7f4d78608388ca7a7fce292d510940fa3c0f5e54"} Sep 18 23:59:41.877354 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:41.877326 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 18 23:59:42.152927 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:42.152903 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 18 23:59:42.154295 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:59:42.154263 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d22a728_bbb0_4ea8_b27a_91823e1d8baa.slice/crio-d3fbf16e59436351448890b9cc3b65990c2cc5c5ae1071b04c8b211b71d8c383 WatchSource:0}: Error finding container d3fbf16e59436351448890b9cc3b65990c2cc5c5ae1071b04c8b211b71d8c383: Status 404 returned error can't find the container with id d3fbf16e59436351448890b9cc3b65990c2cc5c5ae1071b04c8b211b71d8c383 Sep 18 23:59:42.737840 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:42.737779 2568 generic.go:356] "Generic (PLEG): container finished" podID="e0f865cd-5066-4d6e-b11d-aa9ef1dd8702" containerID="9012d73eb415f170b6b66e6e5343a08f60820a7761df829fe2a864d167b29239" exitCode=0 Sep 18 23:59:42.738030 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:42.737881 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ssw9v" event={"ID":"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702","Type":"ContainerDied","Data":"9012d73eb415f170b6b66e6e5343a08f60820a7761df829fe2a864d167b29239"} Sep 18 23:59:42.739215 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:42.739190 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d22a728-bbb0-4ea8-b27a-91823e1d8baa","Type":"ContainerStarted","Data":"d3fbf16e59436351448890b9cc3b65990c2cc5c5ae1071b04c8b211b71d8c383"} Sep 18 23:59:43.744549 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:43.744510 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ssw9v" event={"ID":"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702","Type":"ContainerStarted","Data":"33338a582164e37d938de2b6f504033a08e05620aa2c663820486798342aa670"} Sep 18 23:59:43.744997 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:43.744568 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ssw9v" event={"ID":"e0f865cd-5066-4d6e-b11d-aa9ef1dd8702","Type":"ContainerStarted","Data":"9ff41dfbb04b8c2eac31963194c2ad3d4474809df952d37df06128970ab3c83e"} Sep 18 23:59:43.746203 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:43.746181 2568 generic.go:356] "Generic (PLEG): container finished" podID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerID="53df89c57d659c9a15551bf22bf23e97df8cc10d9e22bac71be28369aef19ae6" exitCode=0 Sep 18 23:59:43.746282 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:43.746216 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d22a728-bbb0-4ea8-b27a-91823e1d8baa","Type":"ContainerDied","Data":"53df89c57d659c9a15551bf22bf23e97df8cc10d9e22bac71be28369aef19ae6"} Sep 18 23:59:43.764711 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:43.764663 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ssw9v" podStartSLOduration=3.769459577 podStartE2EDuration="4.764649044s" totalImagesPullingTime="926.995833ms" totalInitContainerRuntime="68.193634ms" isStatefulPod=true podCreationTimestamp="2026-09-18 23:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:59:43.763177407 +0000 UTC m=+74.916771272" watchObservedRunningTime="2026-09-18 23:59:43.764649044 +0000 UTC m=+74.918242908" Sep 18 23:59:45.675876 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:45.675771 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:45.756777 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:45.756751 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d22a728-bbb0-4ea8-b27a-91823e1d8baa","Type":"ContainerStarted","Data":"72dca2cbaec9753e89aa28e22420f6d5f35a5675b079a0474711a5b675892d78"} Sep 18 23:59:45.756989 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:45.756856 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d22a728-bbb0-4ea8-b27a-91823e1d8baa","Type":"ContainerStarted","Data":"739780a39dbdd30ebd2285041e3b7d29250aeac84f61276f533726f2e7562c63"} Sep 18 23:59:45.756989 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:45.756887 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d22a728-bbb0-4ea8-b27a-91823e1d8baa","Type":"ContainerStarted","Data":"3a31b878b152dca89a0a77636b407d39f62ec55df63cbbfc4d7f6cc38bb1c216"} Sep 18 23:59:45.756989 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:45.756910 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d22a728-bbb0-4ea8-b27a-91823e1d8baa","Type":"ContainerStarted","Data":"b2c4c7b4ca00e180b900fae56e60ef9d40f78a68834fe0c9f3d0b6e447862425"} Sep 18 23:59:45.756989 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:45.756932 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d22a728-bbb0-4ea8-b27a-91823e1d8baa","Type":"ContainerStarted","Data":"a59f2a64a6e8ebd19409943aa76aff3074cd98cf9c9d85ddbaff36222203b799"} Sep 18 23:59:46.763680 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:46.763647 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d22a728-bbb0-4ea8-b27a-91823e1d8baa","Type":"ContainerStarted","Data":"7a232522ae3806a6cbcc0483e9d407f09055c0811cabaaaaa99535b678c9daa2"} Sep 18 23:59:46.793712 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:46.793658 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.599379994 podStartE2EDuration="6.793641684s" totalImagesPullingTime="3.177499998s" totalInitContainerRuntime="16.761692ms" isStatefulPod=false podCreationTimestamp="2026-09-18 23:59:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 23:59:46.791741249 +0000 UTC m=+77.945335118" watchObservedRunningTime="2026-09-18 23:59:46.793641684 +0000 UTC m=+77.947235548" Sep 18 23:59:50.690045 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:50.689907 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76584b8696-td9lt" podUID="85f8145e-e418-47cc-9dc7-f8501c9a8e0f" containerName="registry" containerID="cri-o://18c5f41522a3f6ddc05b69ac00a0e06ef13248a1c398c219072851d39dd5a8ca" gracePeriod=30 Sep 18 23:59:50.926789 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:50.926768 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:51.006459 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.006389 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-ca-trust-extracted\") pod \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " Sep 18 23:59:51.006459 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.006429 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-trusted-ca\") pod \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " Sep 18 23:59:51.006459 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.006456 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-installation-pull-secrets\") pod \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " Sep 18 23:59:51.006720 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.006478 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jv82w\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-kube-api-access-jv82w\") pod \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " Sep 18 23:59:51.006720 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.006493 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-certificates\") pod \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " Sep 18 23:59:51.006720 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.006561 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-bound-sa-token\") pod \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " Sep 18 23:59:51.006720 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.006639 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-image-registry-private-configuration\") pod \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " Sep 18 23:59:51.006720 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.006678 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls\") pod \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\" (UID: \"85f8145e-e418-47cc-9dc7-f8501c9a8e0f\") " Sep 18 23:59:51.007007 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.006874 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "85f8145e-e418-47cc-9dc7-f8501c9a8e0f" (UID: "85f8145e-e418-47cc-9dc7-f8501c9a8e0f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 23:59:51.007114 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.007088 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "85f8145e-e418-47cc-9dc7-f8501c9a8e0f" (UID: "85f8145e-e418-47cc-9dc7-f8501c9a8e0f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 23:59:51.009304 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.009272 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "85f8145e-e418-47cc-9dc7-f8501c9a8e0f" (UID: "85f8145e-e418-47cc-9dc7-f8501c9a8e0f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 23:59:51.009416 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.009327 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-kube-api-access-jv82w" (OuterVolumeSpecName: "kube-api-access-jv82w") pod "85f8145e-e418-47cc-9dc7-f8501c9a8e0f" (UID: "85f8145e-e418-47cc-9dc7-f8501c9a8e0f"). InnerVolumeSpecName "kube-api-access-jv82w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 23:59:51.009416 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.009361 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "85f8145e-e418-47cc-9dc7-f8501c9a8e0f" (UID: "85f8145e-e418-47cc-9dc7-f8501c9a8e0f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 23:59:51.009516 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.009449 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "85f8145e-e418-47cc-9dc7-f8501c9a8e0f" (UID: "85f8145e-e418-47cc-9dc7-f8501c9a8e0f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 23:59:51.009516 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.009493 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "85f8145e-e418-47cc-9dc7-f8501c9a8e0f" (UID: "85f8145e-e418-47cc-9dc7-f8501c9a8e0f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 23:59:51.015126 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.015104 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "85f8145e-e418-47cc-9dc7-f8501c9a8e0f" (UID: "85f8145e-e418-47cc-9dc7-f8501c9a8e0f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 23:59:51.107871 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.107829 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-image-registry-private-configuration\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 18 23:59:51.107871 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.107867 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-tls\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 18 23:59:51.107871 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.107877 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-ca-trust-extracted\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 18 23:59:51.107871 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.107885 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-trusted-ca\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 18 23:59:51.108078 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.107894 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-installation-pull-secrets\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 18 23:59:51.108078 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.107902 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv82w\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-kube-api-access-jv82w\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 18 23:59:51.108078 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.107910 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-registry-certificates\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 18 23:59:51.108078 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.107918 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85f8145e-e418-47cc-9dc7-f8501c9a8e0f-bound-sa-token\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 18 23:59:51.781540 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.781453 2568 generic.go:356] "Generic (PLEG): container finished" podID="85f8145e-e418-47cc-9dc7-f8501c9a8e0f" containerID="18c5f41522a3f6ddc05b69ac00a0e06ef13248a1c398c219072851d39dd5a8ca" exitCode=0 Sep 18 23:59:51.781540 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.781496 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76584b8696-td9lt" Sep 18 23:59:51.781540 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.781520 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76584b8696-td9lt" event={"ID":"85f8145e-e418-47cc-9dc7-f8501c9a8e0f","Type":"ContainerDied","Data":"18c5f41522a3f6ddc05b69ac00a0e06ef13248a1c398c219072851d39dd5a8ca"} Sep 18 23:59:51.782048 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.781558 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76584b8696-td9lt" event={"ID":"85f8145e-e418-47cc-9dc7-f8501c9a8e0f","Type":"ContainerDied","Data":"2f239e12044b1d1d3fb1b6dc7ba883d266a49ee1a70e9ac147c83fd6f1d9527e"} Sep 18 23:59:51.782048 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.781577 2568 scope.go:117] "RemoveContainer" containerID="18c5f41522a3f6ddc05b69ac00a0e06ef13248a1c398c219072851d39dd5a8ca" Sep 18 23:59:51.789682 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.789667 2568 scope.go:117] "RemoveContainer" containerID="18c5f41522a3f6ddc05b69ac00a0e06ef13248a1c398c219072851d39dd5a8ca" Sep 18 23:59:51.789987 ip-10-0-129-146 kubenswrapper[2568]: E0918 23:59:51.789967 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18c5f41522a3f6ddc05b69ac00a0e06ef13248a1c398c219072851d39dd5a8ca\": container with ID starting with 18c5f41522a3f6ddc05b69ac00a0e06ef13248a1c398c219072851d39dd5a8ca not found: ID does not exist" containerID="18c5f41522a3f6ddc05b69ac00a0e06ef13248a1c398c219072851d39dd5a8ca" Sep 18 23:59:51.790042 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.789994 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18c5f41522a3f6ddc05b69ac00a0e06ef13248a1c398c219072851d39dd5a8ca"} err="failed to get container status \"18c5f41522a3f6ddc05b69ac00a0e06ef13248a1c398c219072851d39dd5a8ca\": rpc error: code = NotFound desc = could not find container \"18c5f41522a3f6ddc05b69ac00a0e06ef13248a1c398c219072851d39dd5a8ca\": container with ID starting with 18c5f41522a3f6ddc05b69ac00a0e06ef13248a1c398c219072851d39dd5a8ca not found: ID does not exist" Sep 18 23:59:51.798444 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.798421 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76584b8696-td9lt"] Sep 18 23:59:51.801743 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:51.801723 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76584b8696-td9lt"] Sep 18 23:59:52.748367 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:52.748333 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6594f4c75-p8mhg"] Sep 18 23:59:52.748610 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:52.748598 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85f8145e-e418-47cc-9dc7-f8501c9a8e0f" containerName="registry" Sep 18 23:59:52.748662 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:52.748611 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="85f8145e-e418-47cc-9dc7-f8501c9a8e0f" containerName="registry" Sep 18 23:59:52.748662 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:52.748652 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="85f8145e-e418-47cc-9dc7-f8501c9a8e0f" containerName="registry" Sep 18 23:59:52.753437 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:52.753420 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-p8mhg" Sep 18 23:59:52.755851 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:52.755830 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 18 23:59:52.755961 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:52.755866 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-b942k\"" Sep 18 23:59:52.755961 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:52.755896 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 18 23:59:52.759655 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:52.759626 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-p8mhg"] Sep 18 23:59:52.820360 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:52.820323 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rlrs\" (UniqueName: \"kubernetes.io/projected/338aaefa-b72b-46dd-976c-e8cb141eeba8-kube-api-access-9rlrs\") pod \"downloads-6594f4c75-p8mhg\" (UID: \"338aaefa-b72b-46dd-976c-e8cb141eeba8\") " pod="openshift-console/downloads-6594f4c75-p8mhg" Sep 18 23:59:52.921423 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:52.921386 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rlrs\" (UniqueName: \"kubernetes.io/projected/338aaefa-b72b-46dd-976c-e8cb141eeba8-kube-api-access-9rlrs\") pod \"downloads-6594f4c75-p8mhg\" (UID: \"338aaefa-b72b-46dd-976c-e8cb141eeba8\") " pod="openshift-console/downloads-6594f4c75-p8mhg" Sep 18 23:59:52.928895 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:52.928876 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rlrs\" (UniqueName: \"kubernetes.io/projected/338aaefa-b72b-46dd-976c-e8cb141eeba8-kube-api-access-9rlrs\") pod \"downloads-6594f4c75-p8mhg\" (UID: \"338aaefa-b72b-46dd-976c-e8cb141eeba8\") " pod="openshift-console/downloads-6594f4c75-p8mhg" Sep 18 23:59:53.063241 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:53.063161 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-p8mhg" Sep 18 23:59:53.335084 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:53.335060 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-p8mhg"] Sep 18 23:59:53.337424 ip-10-0-129-146 kubenswrapper[2568]: W0918 23:59:53.337394 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338aaefa_b72b_46dd_976c_e8cb141eeba8.slice/crio-ce4a976578e68f0665d1c3bc4cbb0f4b775f475b59f7d6163da061b122b6deda WatchSource:0}: Error finding container ce4a976578e68f0665d1c3bc4cbb0f4b775f475b59f7d6163da061b122b6deda: Status 404 returned error can't find the container with id ce4a976578e68f0665d1c3bc4cbb0f4b775f475b59f7d6163da061b122b6deda Sep 18 23:59:53.382067 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:53.382027 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85f8145e-e418-47cc-9dc7-f8501c9a8e0f" path="/var/lib/kubelet/pods/85f8145e-e418-47cc-9dc7-f8501c9a8e0f/volumes" Sep 18 23:59:53.791971 ip-10-0-129-146 kubenswrapper[2568]: I0918 23:59:53.791933 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-p8mhg" event={"ID":"338aaefa-b72b-46dd-976c-e8cb141eeba8","Type":"ContainerStarted","Data":"ce4a976578e68f0665d1c3bc4cbb0f4b775f475b59f7d6163da061b122b6deda"} Sep 19 00:00:00.144548 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:00.144505 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-pruner-29829600-s2s2h"] Sep 19 00:00:00.152577 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:00.152549 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29829600-s2s2h" Sep 19 00:00:00.157810 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:00.157757 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"serviceca\"" Sep 19 00:00:00.158159 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:00.157857 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"pruner-dockercfg-wd8js\"" Sep 19 00:00:00.159821 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:00.159393 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29829600-s2s2h"] Sep 19 00:00:00.279513 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:00.279483 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/30b28e1b-b113-4021-aa68-11e495f07f39-serviceca\") pod \"image-pruner-29829600-s2s2h\" (UID: \"30b28e1b-b113-4021-aa68-11e495f07f39\") " pod="openshift-image-registry/image-pruner-29829600-s2s2h" Sep 19 00:00:00.279667 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:00.279586 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwn5p\" (UniqueName: \"kubernetes.io/projected/30b28e1b-b113-4021-aa68-11e495f07f39-kube-api-access-fwn5p\") pod \"image-pruner-29829600-s2s2h\" (UID: \"30b28e1b-b113-4021-aa68-11e495f07f39\") " pod="openshift-image-registry/image-pruner-29829600-s2s2h" Sep 19 00:00:00.380734 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:00.380693 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwn5p\" (UniqueName: \"kubernetes.io/projected/30b28e1b-b113-4021-aa68-11e495f07f39-kube-api-access-fwn5p\") pod \"image-pruner-29829600-s2s2h\" (UID: \"30b28e1b-b113-4021-aa68-11e495f07f39\") " pod="openshift-image-registry/image-pruner-29829600-s2s2h" Sep 19 00:00:00.380919 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:00.380753 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/30b28e1b-b113-4021-aa68-11e495f07f39-serviceca\") pod \"image-pruner-29829600-s2s2h\" (UID: \"30b28e1b-b113-4021-aa68-11e495f07f39\") " pod="openshift-image-registry/image-pruner-29829600-s2s2h" Sep 19 00:00:00.381506 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:00.381483 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/30b28e1b-b113-4021-aa68-11e495f07f39-serviceca\") pod \"image-pruner-29829600-s2s2h\" (UID: \"30b28e1b-b113-4021-aa68-11e495f07f39\") " pod="openshift-image-registry/image-pruner-29829600-s2s2h" Sep 19 00:00:00.389196 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:00.389167 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwn5p\" (UniqueName: \"kubernetes.io/projected/30b28e1b-b113-4021-aa68-11e495f07f39-kube-api-access-fwn5p\") pod \"image-pruner-29829600-s2s2h\" (UID: \"30b28e1b-b113-4021-aa68-11e495f07f39\") " pod="openshift-image-registry/image-pruner-29829600-s2s2h" Sep 19 00:00:00.486205 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:00.486173 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29829600-s2s2h" Sep 19 00:00:00.761503 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:00.761478 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29829600-s2s2h"] Sep 19 00:00:00.764096 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:00:00.764066 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30b28e1b_b113_4021_aa68_11e495f07f39.slice/crio-2579ed114275b975299b1655ac231f518a45532045d96eca39f6566bfcfb4c1e WatchSource:0}: Error finding container 2579ed114275b975299b1655ac231f518a45532045d96eca39f6566bfcfb4c1e: Status 404 returned error can't find the container with id 2579ed114275b975299b1655ac231f518a45532045d96eca39f6566bfcfb4c1e Sep 19 00:00:00.822951 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:00.822920 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29829600-s2s2h" event={"ID":"30b28e1b-b113-4021-aa68-11e495f07f39","Type":"ContainerStarted","Data":"2579ed114275b975299b1655ac231f518a45532045d96eca39f6566bfcfb4c1e"} Sep 19 00:00:01.828618 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:01.828521 2568 generic.go:356] "Generic (PLEG): container finished" podID="30b28e1b-b113-4021-aa68-11e495f07f39" containerID="724c71fcbbfc3608fac307e104d0c4f6b1178f36411049ec2bc49c884f3b2018" exitCode=0 Sep 19 00:00:01.829056 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:01.828615 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29829600-s2s2h" event={"ID":"30b28e1b-b113-4021-aa68-11e495f07f39","Type":"ContainerDied","Data":"724c71fcbbfc3608fac307e104d0c4f6b1178f36411049ec2bc49c884f3b2018"} Sep 19 00:00:08.990183 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:08.990150 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29829600-s2s2h" Sep 19 00:00:09.057617 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:09.057589 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/30b28e1b-b113-4021-aa68-11e495f07f39-serviceca\") pod \"30b28e1b-b113-4021-aa68-11e495f07f39\" (UID: \"30b28e1b-b113-4021-aa68-11e495f07f39\") " Sep 19 00:00:09.057744 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:09.057674 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwn5p\" (UniqueName: \"kubernetes.io/projected/30b28e1b-b113-4021-aa68-11e495f07f39-kube-api-access-fwn5p\") pod \"30b28e1b-b113-4021-aa68-11e495f07f39\" (UID: \"30b28e1b-b113-4021-aa68-11e495f07f39\") " Sep 19 00:00:09.057970 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:09.057943 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30b28e1b-b113-4021-aa68-11e495f07f39-serviceca" (OuterVolumeSpecName: "serviceca") pod "30b28e1b-b113-4021-aa68-11e495f07f39" (UID: "30b28e1b-b113-4021-aa68-11e495f07f39"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 00:00:09.059993 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:09.059963 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30b28e1b-b113-4021-aa68-11e495f07f39-kube-api-access-fwn5p" (OuterVolumeSpecName: "kube-api-access-fwn5p") pod "30b28e1b-b113-4021-aa68-11e495f07f39" (UID: "30b28e1b-b113-4021-aa68-11e495f07f39"). InnerVolumeSpecName "kube-api-access-fwn5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 00:00:09.159031 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:09.158996 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwn5p\" (UniqueName: \"kubernetes.io/projected/30b28e1b-b113-4021-aa68-11e495f07f39-kube-api-access-fwn5p\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:00:09.159031 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:09.159025 2568 reconciler_common.go:299] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/30b28e1b-b113-4021-aa68-11e495f07f39-serviceca\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:00:09.867471 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:09.867431 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-p8mhg" event={"ID":"338aaefa-b72b-46dd-976c-e8cb141eeba8","Type":"ContainerStarted","Data":"67850732ffe1dd40932db3d4f55c503717180db1cd952ccaf1f9a3b2550ce838"} Sep 19 00:00:09.867890 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:09.867860 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6594f4c75-p8mhg" Sep 19 00:00:09.870204 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:09.870173 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29829600-s2s2h" event={"ID":"30b28e1b-b113-4021-aa68-11e495f07f39","Type":"ContainerDied","Data":"2579ed114275b975299b1655ac231f518a45532045d96eca39f6566bfcfb4c1e"} Sep 19 00:00:09.870204 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:09.870198 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29829600-s2s2h" Sep 19 00:00:09.870404 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:09.870216 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2579ed114275b975299b1655ac231f518a45532045d96eca39f6566bfcfb4c1e" Sep 19 00:00:09.885329 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:09.885279 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6594f4c75-p8mhg" podStartSLOduration=2.181275052 podStartE2EDuration="17.885258124s" totalImagesPullingTime="15.703983072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 23:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:00:09.883469733 +0000 UTC m=+101.037063598" watchObservedRunningTime="2026-09-19 00:00:09.885258124 +0000 UTC m=+101.038851994" Sep 19 00:00:09.887332 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:09.887306 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6594f4c75-p8mhg" Sep 19 00:00:10.555345 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:10.555314 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j4srb_f1db5e3f-6ee9-4059-bde8-7c257b1f343d/serve-healthcheck-canary/0.log" Sep 19 00:00:11.734624 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:11.734594 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lpqlq" Sep 19 00:00:12.852517 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.852480 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85c85745f5-hb9cv"] Sep 19 00:00:12.853022 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.852890 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30b28e1b-b113-4021-aa68-11e495f07f39" containerName="image-pruner" Sep 19 00:00:12.853022 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.852910 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="30b28e1b-b113-4021-aa68-11e495f07f39" containerName="image-pruner" Sep 19 00:00:12.853022 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.852989 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="30b28e1b-b113-4021-aa68-11e495f07f39" containerName="image-pruner" Sep 19 00:00:12.883901 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.883870 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85c85745f5-hb9cv"] Sep 19 00:00:12.884069 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.883991 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:12.886776 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.886752 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 19 00:00:12.886925 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.886829 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 19 00:00:12.887084 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.887065 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 19 00:00:12.887847 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.887825 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 19 00:00:12.887991 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.887828 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 19 00:00:12.888119 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.887830 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jsngw\"" Sep 19 00:00:12.892280 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.892258 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 19 00:00:12.991729 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.991698 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-oauth-config\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:12.991932 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.991743 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-service-ca\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:12.991932 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.991807 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-trusted-ca-bundle\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:12.991932 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.991833 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-oauth-serving-cert\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:12.991932 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.991877 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6hpr\" (UniqueName: \"kubernetes.io/projected/339f5ec3-4c62-4174-96e1-dbc6a299f8de-kube-api-access-x6hpr\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:12.991932 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.991896 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-config\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:12.991932 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:12.991929 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-serving-cert\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:13.092880 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.092839 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-service-ca\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:13.093068 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.092903 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-trusted-ca-bundle\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:13.093068 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.093015 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-oauth-serving-cert\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:13.093068 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.093053 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6hpr\" (UniqueName: \"kubernetes.io/projected/339f5ec3-4c62-4174-96e1-dbc6a299f8de-kube-api-access-x6hpr\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:13.093222 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.093080 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-config\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:13.093222 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.093115 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-serving-cert\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:13.093433 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.093218 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-oauth-config\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:13.093731 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.093705 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-service-ca\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:13.094374 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.094310 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-config\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:13.094374 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.094320 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-oauth-serving-cert\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:13.094537 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.094505 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-trusted-ca-bundle\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:13.096448 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.096423 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-serving-cert\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:13.096535 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.096447 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-oauth-config\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:13.101576 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.101548 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6hpr\" (UniqueName: \"kubernetes.io/projected/339f5ec3-4c62-4174-96e1-dbc6a299f8de-kube-api-access-x6hpr\") pod \"console-85c85745f5-hb9cv\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:13.195596 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.195559 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:13.511219 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.511186 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85c85745f5-hb9cv"] Sep 19 00:00:13.513976 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:00:13.513949 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod339f5ec3_4c62_4174_96e1_dbc6a299f8de.slice/crio-5e198dc879b005e4c2cd0fb4e5a9efa48ed696158db3fcc16fc7ae7244fbfc8c WatchSource:0}: Error finding container 5e198dc879b005e4c2cd0fb4e5a9efa48ed696158db3fcc16fc7ae7244fbfc8c: Status 404 returned error can't find the container with id 5e198dc879b005e4c2cd0fb4e5a9efa48ed696158db3fcc16fc7ae7244fbfc8c Sep 19 00:00:13.888076 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:13.888033 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85c85745f5-hb9cv" event={"ID":"339f5ec3-4c62-4174-96e1-dbc6a299f8de","Type":"ContainerStarted","Data":"5e198dc879b005e4c2cd0fb4e5a9efa48ed696158db3fcc16fc7ae7244fbfc8c"} Sep 19 00:00:20.923263 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:20.923214 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85c85745f5-hb9cv" event={"ID":"339f5ec3-4c62-4174-96e1-dbc6a299f8de","Type":"ContainerStarted","Data":"3e624310f87a90d602bb12fb6d477345867500d58d314403772c102c1a50e18b"} Sep 19 00:00:20.940622 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:20.940566 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85c85745f5-hb9cv" podStartSLOduration=2.5016287999999998 podStartE2EDuration="8.940549698s" totalImagesPullingTime="6.438920898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:00:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:00:20.939389503 +0000 UTC m=+112.092983376" watchObservedRunningTime="2026-09-19 00:00:20.940549698 +0000 UTC m=+112.094143563" Sep 19 00:00:23.196046 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:23.196016 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:23.196441 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:23.196073 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:23.202019 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:23.201990 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:00:23.937689 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:00:23.937656 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:01:00.127145 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:00.127113 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 19 00:01:00.127623 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:00.127508 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="alertmanager" containerID="cri-o://a59f2a64a6e8ebd19409943aa76aff3074cd98cf9c9d85ddbaff36222203b799" gracePeriod=120 Sep 19 00:01:00.127623 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:00.127542 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="kube-rbac-proxy-metric" containerID="cri-o://72dca2cbaec9753e89aa28e22420f6d5f35a5675b079a0474711a5b675892d78" gracePeriod=120 Sep 19 00:01:00.127746 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:00.127592 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="prom-label-proxy" containerID="cri-o://7a232522ae3806a6cbcc0483e9d407f09055c0811cabaaaaa99535b678c9daa2" gracePeriod=120 Sep 19 00:01:00.127746 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:00.127615 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="kube-rbac-proxy-web" containerID="cri-o://3a31b878b152dca89a0a77636b407d39f62ec55df63cbbfc4d7f6cc38bb1c216" gracePeriod=120 Sep 19 00:01:00.127746 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:00.127650 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="config-reloader" containerID="cri-o://b2c4c7b4ca00e180b900fae56e60ef9d40f78a68834fe0c9f3d0b6e447862425" gracePeriod=120 Sep 19 00:01:00.127925 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:00.127750 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="kube-rbac-proxy" containerID="cri-o://739780a39dbdd30ebd2285041e3b7d29250aeac84f61276f533726f2e7562c63" gracePeriod=120 Sep 19 00:01:01.068238 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.068207 2568 generic.go:356] "Generic (PLEG): container finished" podID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerID="7a232522ae3806a6cbcc0483e9d407f09055c0811cabaaaaa99535b678c9daa2" exitCode=0 Sep 19 00:01:01.068238 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.068231 2568 generic.go:356] "Generic (PLEG): container finished" podID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerID="72dca2cbaec9753e89aa28e22420f6d5f35a5675b079a0474711a5b675892d78" exitCode=0 Sep 19 00:01:01.068238 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.068238 2568 generic.go:356] "Generic (PLEG): container finished" podID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerID="739780a39dbdd30ebd2285041e3b7d29250aeac84f61276f533726f2e7562c63" exitCode=0 Sep 19 00:01:01.068238 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.068243 2568 generic.go:356] "Generic (PLEG): container finished" podID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerID="b2c4c7b4ca00e180b900fae56e60ef9d40f78a68834fe0c9f3d0b6e447862425" exitCode=0 Sep 19 00:01:01.068238 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.068248 2568 generic.go:356] "Generic (PLEG): container finished" podID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerID="a59f2a64a6e8ebd19409943aa76aff3074cd98cf9c9d85ddbaff36222203b799" exitCode=0 Sep 19 00:01:01.068521 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.068270 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d22a728-bbb0-4ea8-b27a-91823e1d8baa","Type":"ContainerDied","Data":"7a232522ae3806a6cbcc0483e9d407f09055c0811cabaaaaa99535b678c9daa2"} Sep 19 00:01:01.068521 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.068330 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d22a728-bbb0-4ea8-b27a-91823e1d8baa","Type":"ContainerDied","Data":"72dca2cbaec9753e89aa28e22420f6d5f35a5675b079a0474711a5b675892d78"} Sep 19 00:01:01.068521 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.068359 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d22a728-bbb0-4ea8-b27a-91823e1d8baa","Type":"ContainerDied","Data":"739780a39dbdd30ebd2285041e3b7d29250aeac84f61276f533726f2e7562c63"} Sep 19 00:01:01.068521 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.068384 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d22a728-bbb0-4ea8-b27a-91823e1d8baa","Type":"ContainerDied","Data":"b2c4c7b4ca00e180b900fae56e60ef9d40f78a68834fe0c9f3d0b6e447862425"} Sep 19 00:01:01.068521 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.068409 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d22a728-bbb0-4ea8-b27a-91823e1d8baa","Type":"ContainerDied","Data":"a59f2a64a6e8ebd19409943aa76aff3074cd98cf9c9d85ddbaff36222203b799"} Sep 19 00:01:01.401475 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.401444 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:01.463196 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.463157 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-main-tls\") pod \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " Sep 19 00:01:01.463346 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.463206 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-tls-assets\") pod \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " Sep 19 00:01:01.463346 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.463234 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-alertmanager-trusted-ca-bundle\") pod \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " Sep 19 00:01:01.463438 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.463397 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-alertmanager-main-db\") pod \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " Sep 19 00:01:01.463500 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.463436 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-config-volume\") pod \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " Sep 19 00:01:01.463500 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.463476 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-metrics-client-ca\") pod \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " Sep 19 00:01:01.463598 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.463501 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sptx7\" (UniqueName: \"kubernetes.io/projected/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-kube-api-access-sptx7\") pod \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " Sep 19 00:01:01.463598 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.463537 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " Sep 19 00:01:01.463598 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.463569 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-config-out\") pod \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " Sep 19 00:01:01.463598 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.463596 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-web-config\") pod \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " Sep 19 00:01:01.463829 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.463627 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy-web\") pod \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " Sep 19 00:01:01.463829 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.463673 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy\") pod \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " Sep 19 00:01:01.463829 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.463669 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "2d22a728-bbb0-4ea8-b27a-91823e1d8baa" (UID: "2d22a728-bbb0-4ea8-b27a-91823e1d8baa"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 00:01:01.463829 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.463699 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-cluster-tls-config\") pod \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\" (UID: \"2d22a728-bbb0-4ea8-b27a-91823e1d8baa\") " Sep 19 00:01:01.463829 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.463697 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "2d22a728-bbb0-4ea8-b27a-91823e1d8baa" (UID: "2d22a728-bbb0-4ea8-b27a-91823e1d8baa"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 00:01:01.464084 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.464058 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "2d22a728-bbb0-4ea8-b27a-91823e1d8baa" (UID: "2d22a728-bbb0-4ea8-b27a-91823e1d8baa"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 00:01:01.464175 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.464156 2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-alertmanager-main-db\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:01.464239 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.464180 2568 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-metrics-client-ca\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:01.464239 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.464196 2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:01.466491 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.466469 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2d22a728-bbb0-4ea8-b27a-91823e1d8baa" (UID: "2d22a728-bbb0-4ea8-b27a-91823e1d8baa"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 00:01:01.467262 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.467225 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "2d22a728-bbb0-4ea8-b27a-91823e1d8baa" (UID: "2d22a728-bbb0-4ea8-b27a-91823e1d8baa"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 00:01:01.467497 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.467468 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "2d22a728-bbb0-4ea8-b27a-91823e1d8baa" (UID: "2d22a728-bbb0-4ea8-b27a-91823e1d8baa"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 00:01:01.468131 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.468096 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "2d22a728-bbb0-4ea8-b27a-91823e1d8baa" (UID: "2d22a728-bbb0-4ea8-b27a-91823e1d8baa"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 00:01:01.468664 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.468618 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-kube-api-access-sptx7" (OuterVolumeSpecName: "kube-api-access-sptx7") pod "2d22a728-bbb0-4ea8-b27a-91823e1d8baa" (UID: "2d22a728-bbb0-4ea8-b27a-91823e1d8baa"). InnerVolumeSpecName "kube-api-access-sptx7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 00:01:01.468890 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.468864 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-config-out" (OuterVolumeSpecName: "config-out") pod "2d22a728-bbb0-4ea8-b27a-91823e1d8baa" (UID: "2d22a728-bbb0-4ea8-b27a-91823e1d8baa"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 00:01:01.468989 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.468966 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "2d22a728-bbb0-4ea8-b27a-91823e1d8baa" (UID: "2d22a728-bbb0-4ea8-b27a-91823e1d8baa"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 00:01:01.469311 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.469281 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-config-volume" (OuterVolumeSpecName: "config-volume") pod "2d22a728-bbb0-4ea8-b27a-91823e1d8baa" (UID: "2d22a728-bbb0-4ea8-b27a-91823e1d8baa"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 00:01:01.472667 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.472639 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "2d22a728-bbb0-4ea8-b27a-91823e1d8baa" (UID: "2d22a728-bbb0-4ea8-b27a-91823e1d8baa"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 00:01:01.477969 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.477945 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-web-config" (OuterVolumeSpecName: "web-config") pod "2d22a728-bbb0-4ea8-b27a-91823e1d8baa" (UID: "2d22a728-bbb0-4ea8-b27a-91823e1d8baa"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 00:01:01.565291 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.565256 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-main-tls\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:01.565291 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.565288 2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-tls-assets\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:01.565291 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.565297 2568 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-config-volume\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:01.565509 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.565306 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sptx7\" (UniqueName: \"kubernetes.io/projected/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-kube-api-access-sptx7\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:01.565509 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.565316 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:01.565509 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.565326 2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-config-out\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:01.565509 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.565335 2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-web-config\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:01.565509 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.565343 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:01.565509 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.565352 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:01.565509 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:01.565361 2568 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2d22a728-bbb0-4ea8-b27a-91823e1d8baa-cluster-tls-config\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:02.074826 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.074781 2568 generic.go:356] "Generic (PLEG): container finished" podID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerID="3a31b878b152dca89a0a77636b407d39f62ec55df63cbbfc4d7f6cc38bb1c216" exitCode=0 Sep 19 00:01:02.075001 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.074829 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d22a728-bbb0-4ea8-b27a-91823e1d8baa","Type":"ContainerDied","Data":"3a31b878b152dca89a0a77636b407d39f62ec55df63cbbfc4d7f6cc38bb1c216"} Sep 19 00:01:02.075001 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.074894 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.075001 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.074912 2568 scope.go:117] "RemoveContainer" containerID="7a232522ae3806a6cbcc0483e9d407f09055c0811cabaaaaa99535b678c9daa2" Sep 19 00:01:02.075193 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.074897 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d22a728-bbb0-4ea8-b27a-91823e1d8baa","Type":"ContainerDied","Data":"d3fbf16e59436351448890b9cc3b65990c2cc5c5ae1071b04c8b211b71d8c383"} Sep 19 00:01:02.083183 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.083155 2568 scope.go:117] "RemoveContainer" containerID="72dca2cbaec9753e89aa28e22420f6d5f35a5675b079a0474711a5b675892d78" Sep 19 00:01:02.090507 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.090490 2568 scope.go:117] "RemoveContainer" containerID="739780a39dbdd30ebd2285041e3b7d29250aeac84f61276f533726f2e7562c63" Sep 19 00:01:02.097482 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.097461 2568 scope.go:117] "RemoveContainer" containerID="3a31b878b152dca89a0a77636b407d39f62ec55df63cbbfc4d7f6cc38bb1c216" Sep 19 00:01:02.097582 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.097492 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 19 00:01:02.104201 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.104173 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 19 00:01:02.104920 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.104906 2568 scope.go:117] "RemoveContainer" containerID="b2c4c7b4ca00e180b900fae56e60ef9d40f78a68834fe0c9f3d0b6e447862425" Sep 19 00:01:02.112159 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.112137 2568 scope.go:117] "RemoveContainer" containerID="a59f2a64a6e8ebd19409943aa76aff3074cd98cf9c9d85ddbaff36222203b799" Sep 19 00:01:02.119637 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.119614 2568 scope.go:117] "RemoveContainer" containerID="53df89c57d659c9a15551bf22bf23e97df8cc10d9e22bac71be28369aef19ae6" Sep 19 00:01:02.127524 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127505 2568 scope.go:117] "RemoveContainer" containerID="7a232522ae3806a6cbcc0483e9d407f09055c0811cabaaaaa99535b678c9daa2" Sep 19 00:01:02.127593 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127560 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 19 00:01:02.127871 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127857 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="init-config-reloader" Sep 19 00:01:02.127917 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127878 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="init-config-reloader" Sep 19 00:01:02.127917 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127886 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="alertmanager" Sep 19 00:01:02.127917 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127891 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="alertmanager" Sep 19 00:01:02.127917 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127900 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="kube-rbac-proxy" Sep 19 00:01:02.127917 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127906 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="kube-rbac-proxy" Sep 19 00:01:02.127917 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127914 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="kube-rbac-proxy-metric" Sep 19 00:01:02.127917 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127919 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="kube-rbac-proxy-metric" Sep 19 00:01:02.128125 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127931 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="kube-rbac-proxy-web" Sep 19 00:01:02.128125 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127936 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="kube-rbac-proxy-web" Sep 19 00:01:02.128125 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:01:02.127858 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a232522ae3806a6cbcc0483e9d407f09055c0811cabaaaaa99535b678c9daa2\": container with ID starting with 7a232522ae3806a6cbcc0483e9d407f09055c0811cabaaaaa99535b678c9daa2 not found: ID does not exist" containerID="7a232522ae3806a6cbcc0483e9d407f09055c0811cabaaaaa99535b678c9daa2" Sep 19 00:01:02.128125 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127944 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="prom-label-proxy" Sep 19 00:01:02.128125 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127949 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="prom-label-proxy" Sep 19 00:01:02.128125 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127957 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="config-reloader" Sep 19 00:01:02.128125 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127962 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="config-reloader" Sep 19 00:01:02.128125 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127958 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a232522ae3806a6cbcc0483e9d407f09055c0811cabaaaaa99535b678c9daa2"} err="failed to get container status \"7a232522ae3806a6cbcc0483e9d407f09055c0811cabaaaaa99535b678c9daa2\": rpc error: code = NotFound desc = could not find container \"7a232522ae3806a6cbcc0483e9d407f09055c0811cabaaaaa99535b678c9daa2\": container with ID starting with 7a232522ae3806a6cbcc0483e9d407f09055c0811cabaaaaa99535b678c9daa2 not found: ID does not exist" Sep 19 00:01:02.128125 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.127983 2568 scope.go:117] "RemoveContainer" containerID="72dca2cbaec9753e89aa28e22420f6d5f35a5675b079a0474711a5b675892d78" Sep 19 00:01:02.128125 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.128001 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="kube-rbac-proxy-web" Sep 19 00:01:02.128125 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.128009 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="alertmanager" Sep 19 00:01:02.128125 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.128015 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="kube-rbac-proxy-metric" Sep 19 00:01:02.128125 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.128021 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="config-reloader" Sep 19 00:01:02.128125 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.128026 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="kube-rbac-proxy" Sep 19 00:01:02.128125 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.128032 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" containerName="prom-label-proxy" Sep 19 00:01:02.128667 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:01:02.128272 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72dca2cbaec9753e89aa28e22420f6d5f35a5675b079a0474711a5b675892d78\": container with ID starting with 72dca2cbaec9753e89aa28e22420f6d5f35a5675b079a0474711a5b675892d78 not found: ID does not exist" containerID="72dca2cbaec9753e89aa28e22420f6d5f35a5675b079a0474711a5b675892d78" Sep 19 00:01:02.128667 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.128299 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72dca2cbaec9753e89aa28e22420f6d5f35a5675b079a0474711a5b675892d78"} err="failed to get container status \"72dca2cbaec9753e89aa28e22420f6d5f35a5675b079a0474711a5b675892d78\": rpc error: code = NotFound desc = could not find container \"72dca2cbaec9753e89aa28e22420f6d5f35a5675b079a0474711a5b675892d78\": container with ID starting with 72dca2cbaec9753e89aa28e22420f6d5f35a5675b079a0474711a5b675892d78 not found: ID does not exist" Sep 19 00:01:02.128667 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.128317 2568 scope.go:117] "RemoveContainer" containerID="739780a39dbdd30ebd2285041e3b7d29250aeac84f61276f533726f2e7562c63" Sep 19 00:01:02.128667 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:01:02.128590 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"739780a39dbdd30ebd2285041e3b7d29250aeac84f61276f533726f2e7562c63\": container with ID starting with 739780a39dbdd30ebd2285041e3b7d29250aeac84f61276f533726f2e7562c63 not found: ID does not exist" containerID="739780a39dbdd30ebd2285041e3b7d29250aeac84f61276f533726f2e7562c63" Sep 19 00:01:02.128667 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.128607 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"739780a39dbdd30ebd2285041e3b7d29250aeac84f61276f533726f2e7562c63"} err="failed to get container status \"739780a39dbdd30ebd2285041e3b7d29250aeac84f61276f533726f2e7562c63\": rpc error: code = NotFound desc = could not find container \"739780a39dbdd30ebd2285041e3b7d29250aeac84f61276f533726f2e7562c63\": container with ID starting with 739780a39dbdd30ebd2285041e3b7d29250aeac84f61276f533726f2e7562c63 not found: ID does not exist" Sep 19 00:01:02.128667 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.128621 2568 scope.go:117] "RemoveContainer" containerID="3a31b878b152dca89a0a77636b407d39f62ec55df63cbbfc4d7f6cc38bb1c216" Sep 19 00:01:02.128930 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:01:02.128905 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a31b878b152dca89a0a77636b407d39f62ec55df63cbbfc4d7f6cc38bb1c216\": container with ID starting with 3a31b878b152dca89a0a77636b407d39f62ec55df63cbbfc4d7f6cc38bb1c216 not found: ID does not exist" containerID="3a31b878b152dca89a0a77636b407d39f62ec55df63cbbfc4d7f6cc38bb1c216" Sep 19 00:01:02.128965 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.128931 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a31b878b152dca89a0a77636b407d39f62ec55df63cbbfc4d7f6cc38bb1c216"} err="failed to get container status \"3a31b878b152dca89a0a77636b407d39f62ec55df63cbbfc4d7f6cc38bb1c216\": rpc error: code = NotFound desc = could not find container \"3a31b878b152dca89a0a77636b407d39f62ec55df63cbbfc4d7f6cc38bb1c216\": container with ID starting with 3a31b878b152dca89a0a77636b407d39f62ec55df63cbbfc4d7f6cc38bb1c216 not found: ID does not exist" Sep 19 00:01:02.128965 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.128943 2568 scope.go:117] "RemoveContainer" containerID="b2c4c7b4ca00e180b900fae56e60ef9d40f78a68834fe0c9f3d0b6e447862425" Sep 19 00:01:02.129172 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:01:02.129156 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2c4c7b4ca00e180b900fae56e60ef9d40f78a68834fe0c9f3d0b6e447862425\": container with ID starting with b2c4c7b4ca00e180b900fae56e60ef9d40f78a68834fe0c9f3d0b6e447862425 not found: ID does not exist" containerID="b2c4c7b4ca00e180b900fae56e60ef9d40f78a68834fe0c9f3d0b6e447862425" Sep 19 00:01:02.129214 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.129176 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2c4c7b4ca00e180b900fae56e60ef9d40f78a68834fe0c9f3d0b6e447862425"} err="failed to get container status \"b2c4c7b4ca00e180b900fae56e60ef9d40f78a68834fe0c9f3d0b6e447862425\": rpc error: code = NotFound desc = could not find container \"b2c4c7b4ca00e180b900fae56e60ef9d40f78a68834fe0c9f3d0b6e447862425\": container with ID starting with b2c4c7b4ca00e180b900fae56e60ef9d40f78a68834fe0c9f3d0b6e447862425 not found: ID does not exist" Sep 19 00:01:02.129214 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.129187 2568 scope.go:117] "RemoveContainer" containerID="a59f2a64a6e8ebd19409943aa76aff3074cd98cf9c9d85ddbaff36222203b799" Sep 19 00:01:02.129393 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:01:02.129378 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a59f2a64a6e8ebd19409943aa76aff3074cd98cf9c9d85ddbaff36222203b799\": container with ID starting with a59f2a64a6e8ebd19409943aa76aff3074cd98cf9c9d85ddbaff36222203b799 not found: ID does not exist" containerID="a59f2a64a6e8ebd19409943aa76aff3074cd98cf9c9d85ddbaff36222203b799" Sep 19 00:01:02.129438 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.129395 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a59f2a64a6e8ebd19409943aa76aff3074cd98cf9c9d85ddbaff36222203b799"} err="failed to get container status \"a59f2a64a6e8ebd19409943aa76aff3074cd98cf9c9d85ddbaff36222203b799\": rpc error: code = NotFound desc = could not find container \"a59f2a64a6e8ebd19409943aa76aff3074cd98cf9c9d85ddbaff36222203b799\": container with ID starting with a59f2a64a6e8ebd19409943aa76aff3074cd98cf9c9d85ddbaff36222203b799 not found: ID does not exist" Sep 19 00:01:02.129438 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.129407 2568 scope.go:117] "RemoveContainer" containerID="53df89c57d659c9a15551bf22bf23e97df8cc10d9e22bac71be28369aef19ae6" Sep 19 00:01:02.129648 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:01:02.129630 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53df89c57d659c9a15551bf22bf23e97df8cc10d9e22bac71be28369aef19ae6\": container with ID starting with 53df89c57d659c9a15551bf22bf23e97df8cc10d9e22bac71be28369aef19ae6 not found: ID does not exist" containerID="53df89c57d659c9a15551bf22bf23e97df8cc10d9e22bac71be28369aef19ae6" Sep 19 00:01:02.129694 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.129651 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53df89c57d659c9a15551bf22bf23e97df8cc10d9e22bac71be28369aef19ae6"} err="failed to get container status \"53df89c57d659c9a15551bf22bf23e97df8cc10d9e22bac71be28369aef19ae6\": rpc error: code = NotFound desc = could not find container \"53df89c57d659c9a15551bf22bf23e97df8cc10d9e22bac71be28369aef19ae6\": container with ID starting with 53df89c57d659c9a15551bf22bf23e97df8cc10d9e22bac71be28369aef19ae6 not found: ID does not exist" Sep 19 00:01:02.131490 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.131475 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.134110 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.134001 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 19 00:01:02.134110 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.134004 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 19 00:01:02.134424 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.134406 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 19 00:01:02.134543 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.134410 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 19 00:01:02.134543 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.134487 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jrj22\"" Sep 19 00:01:02.134543 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.134529 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 19 00:01:02.134715 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.134656 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 19 00:01:02.134841 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.134824 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 19 00:01:02.134982 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.134952 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 19 00:01:02.139595 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.139577 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 19 00:01:02.144278 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.144256 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 19 00:01:02.169695 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.169663 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.169923 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.169712 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.169923 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.169782 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.169923 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.169849 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.169923 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.169876 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.169923 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.169906 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.170151 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.169957 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.170151 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.169987 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-config-out\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.170151 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.170002 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.170151 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.170020 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-web-config\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.170151 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.170063 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-config-volume\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.170151 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.170088 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46s6v\" (UniqueName: \"kubernetes.io/projected/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-kube-api-access-46s6v\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.170151 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.170120 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.271529 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.271493 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.271703 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.271540 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.271703 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.271559 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.271703 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.271582 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-config-out\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.271859 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.271717 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.271859 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.271763 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-web-config\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.271859 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.271826 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-config-volume\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.271859 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.271849 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46s6v\" (UniqueName: \"kubernetes.io/projected/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-kube-api-access-46s6v\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.272064 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.271876 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.272064 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.271929 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.272064 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.271962 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.272064 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.272004 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.272064 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.272059 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.272543 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.272517 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.272787 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.272731 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.273632 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.273280 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.274960 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.274935 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-config-out\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.275117 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.275072 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.275360 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.275324 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.275360 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.275338 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.275641 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.275607 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.276043 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.276020 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-web-config\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.276148 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.276127 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.276372 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.276357 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.276808 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.276778 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-config-volume\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.281658 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.281633 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46s6v\" (UniqueName: \"kubernetes.io/projected/9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf-kube-api-access-46s6v\") pod \"alertmanager-main-0\" (UID: \"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.440979 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.440943 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 19 00:01:02.718276 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:02.718247 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 19 00:01:02.719398 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:01:02.719375 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b6d489a_1018_4cdc_a5e4_ca0038ff9bcf.slice/crio-ae5480ab6f01635fb52fbd99f81d3f51ff336ea6966033eea85538ada9efd581 WatchSource:0}: Error finding container ae5480ab6f01635fb52fbd99f81d3f51ff336ea6966033eea85538ada9efd581: Status 404 returned error can't find the container with id ae5480ab6f01635fb52fbd99f81d3f51ff336ea6966033eea85538ada9efd581 Sep 19 00:01:03.081321 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:03.081245 2568 generic.go:356] "Generic (PLEG): container finished" podID="9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf" containerID="fe84718a04d2c8a14dbd63ed49520f9f2fda702af038e2fd1192bf0b09d735f8" exitCode=0 Sep 19 00:01:03.081321 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:03.081285 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf","Type":"ContainerDied","Data":"fe84718a04d2c8a14dbd63ed49520f9f2fda702af038e2fd1192bf0b09d735f8"} Sep 19 00:01:03.081490 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:03.081334 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf","Type":"ContainerStarted","Data":"ae5480ab6f01635fb52fbd99f81d3f51ff336ea6966033eea85538ada9efd581"} Sep 19 00:01:03.379756 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:03.379730 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d22a728-bbb0-4ea8-b27a-91823e1d8baa" path="/var/lib/kubelet/pods/2d22a728-bbb0-4ea8-b27a-91823e1d8baa/volumes" Sep 19 00:01:04.087779 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.087746 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf","Type":"ContainerStarted","Data":"715de43f0a08bdfb62d40eac8225e64dfcade72675cceabc7a5df95e65c1b43a"} Sep 19 00:01:04.088131 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.087828 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf","Type":"ContainerStarted","Data":"ac90e602188fd55087d10f5db2a4e9380a0f00ea97c8bbdbf3d4661c78eee067"} Sep 19 00:01:04.088131 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.087853 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf","Type":"ContainerStarted","Data":"a01e118ffaa192999b4b59cf5e7eec7d2326de7a96d5d60475a3c751beac6c19"} Sep 19 00:01:04.088131 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.087878 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf","Type":"ContainerStarted","Data":"c5d44f920f745a884a6c7cb34ade26dc39b904e2f95ca2e06760d68f8524866d"} Sep 19 00:01:04.088131 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.087901 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf","Type":"ContainerStarted","Data":"6cc56daf50794d1737324d69bd3de5e959d66ec7466facc3b2da489636387eef"} Sep 19 00:01:04.088131 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.087925 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf","Type":"ContainerStarted","Data":"b921abf41d6927008e8561539d1097a10800a91db65f03b02313b59c0cd97921"} Sep 19 00:01:04.115788 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.115733 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.099959933 podStartE2EDuration="2.11571746s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.757527ms" isStatefulPod=false podCreationTimestamp="2026-09-19 00:01:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:01:04.113427204 +0000 UTC m=+155.267021069" watchObservedRunningTime="2026-09-19 00:01:04.11571746 +0000 UTC m=+155.269311359" Sep 19 00:01:04.159600 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.159564 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q"] Sep 19 00:01:04.161825 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.161784 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.164221 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.164197 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-q89r5\"" Sep 19 00:01:04.164221 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.164213 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 19 00:01:04.164488 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.164314 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 19 00:01:04.164488 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.164328 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 19 00:01:04.164488 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.164314 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 19 00:01:04.164646 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.164629 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 19 00:01:04.169820 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.169775 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 19 00:01:04.178367 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.178337 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q"] Sep 19 00:01:04.288912 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.288856 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0e96afe-381b-42f2-915d-53555a0f6f1f-serving-certs-ca-bundle\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.288912 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.288911 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d0e96afe-381b-42f2-915d-53555a0f6f1f-federate-client-tls\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.289165 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.288939 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d0e96afe-381b-42f2-915d-53555a0f6f1f-secret-telemeter-client\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.289165 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.288973 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d0e96afe-381b-42f2-915d-53555a0f6f1f-telemeter-client-tls\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.289165 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.289034 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0e96afe-381b-42f2-915d-53555a0f6f1f-metrics-client-ca\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.289165 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.289064 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0e96afe-381b-42f2-915d-53555a0f6f1f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.289165 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.289102 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt74n\" (UniqueName: \"kubernetes.io/projected/d0e96afe-381b-42f2-915d-53555a0f6f1f-kube-api-access-mt74n\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.289165 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.289140 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0e96afe-381b-42f2-915d-53555a0f6f1f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.390260 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.390231 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt74n\" (UniqueName: \"kubernetes.io/projected/d0e96afe-381b-42f2-915d-53555a0f6f1f-kube-api-access-mt74n\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.390260 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.390265 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0e96afe-381b-42f2-915d-53555a0f6f1f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.390505 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.390292 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0e96afe-381b-42f2-915d-53555a0f6f1f-serving-certs-ca-bundle\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.390505 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.390316 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d0e96afe-381b-42f2-915d-53555a0f6f1f-federate-client-tls\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.390505 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.390346 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d0e96afe-381b-42f2-915d-53555a0f6f1f-secret-telemeter-client\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.390505 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.390377 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d0e96afe-381b-42f2-915d-53555a0f6f1f-telemeter-client-tls\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.390505 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.390433 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0e96afe-381b-42f2-915d-53555a0f6f1f-metrics-client-ca\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.390505 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.390464 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0e96afe-381b-42f2-915d-53555a0f6f1f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.391133 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.391102 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0e96afe-381b-42f2-915d-53555a0f6f1f-serving-certs-ca-bundle\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.391133 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.391125 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0e96afe-381b-42f2-915d-53555a0f6f1f-metrics-client-ca\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.391531 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.391509 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0e96afe-381b-42f2-915d-53555a0f6f1f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.393211 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.393186 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0e96afe-381b-42f2-915d-53555a0f6f1f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.393306 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.393193 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d0e96afe-381b-42f2-915d-53555a0f6f1f-telemeter-client-tls\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.393360 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.393341 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d0e96afe-381b-42f2-915d-53555a0f6f1f-federate-client-tls\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.393518 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.393496 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d0e96afe-381b-42f2-915d-53555a0f6f1f-secret-telemeter-client\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.397157 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.397136 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt74n\" (UniqueName: \"kubernetes.io/projected/d0e96afe-381b-42f2-915d-53555a0f6f1f-kube-api-access-mt74n\") pod \"telemeter-client-7b79cc9d87-c9l2q\" (UID: \"d0e96afe-381b-42f2-915d-53555a0f6f1f\") " pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.471436 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.471402 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" Sep 19 00:01:04.745621 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:04.745595 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q"] Sep 19 00:01:04.747338 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:01:04.747304 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0e96afe_381b_42f2_915d_53555a0f6f1f.slice/crio-638d9291c2f9ee4d527f6d9aa1b6197b344dd849d686c0bc8fbfdd88b050f54c WatchSource:0}: Error finding container 638d9291c2f9ee4d527f6d9aa1b6197b344dd849d686c0bc8fbfdd88b050f54c: Status 404 returned error can't find the container with id 638d9291c2f9ee4d527f6d9aa1b6197b344dd849d686c0bc8fbfdd88b050f54c Sep 19 00:01:05.092971 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:05.092886 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" event={"ID":"d0e96afe-381b-42f2-915d-53555a0f6f1f","Type":"ContainerStarted","Data":"638d9291c2f9ee4d527f6d9aa1b6197b344dd849d686c0bc8fbfdd88b050f54c"} Sep 19 00:01:08.106708 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:08.106612 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" event={"ID":"d0e96afe-381b-42f2-915d-53555a0f6f1f","Type":"ContainerStarted","Data":"cb6aba10072abaee873909536013fdd4dc759f4a4ac0643d091e838d3a5910dc"} Sep 19 00:01:08.106708 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:08.106673 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" event={"ID":"d0e96afe-381b-42f2-915d-53555a0f6f1f","Type":"ContainerStarted","Data":"fe530f9e58ae3b37bf0c3e7365a51be2fb7e083574cf73bec98f553dab870081"} Sep 19 00:01:08.106708 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:08.106691 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" event={"ID":"d0e96afe-381b-42f2-915d-53555a0f6f1f","Type":"ContainerStarted","Data":"9ac5d0de26c2959f9d2dfa19e6cc7f815082100f0337157c58463932f52e256a"} Sep 19 00:01:08.127885 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:08.127834 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7b79cc9d87-c9l2q" podStartSLOduration=1.165948632 podStartE2EDuration="4.127816479s" totalImagesPullingTime="2.961867847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:01:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:01:08.126148046 +0000 UTC m=+159.279741911" watchObservedRunningTime="2026-09-19 00:01:08.127816479 +0000 UTC m=+159.281410344" Sep 19 00:01:19.078211 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:19.078165 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85c85745f5-hb9cv"] Sep 19 00:01:44.097738 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.097676 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85c85745f5-hb9cv" podUID="339f5ec3-4c62-4174-96e1-dbc6a299f8de" containerName="console" containerID="cri-o://3e624310f87a90d602bb12fb6d477345867500d58d314403772c102c1a50e18b" gracePeriod=15 Sep 19 00:01:44.246364 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.246335 2568 generic.go:356] "Generic (PLEG): container finished" podID="339f5ec3-4c62-4174-96e1-dbc6a299f8de" containerID="3e624310f87a90d602bb12fb6d477345867500d58d314403772c102c1a50e18b" exitCode=0 Sep 19 00:01:44.246520 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.246404 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85c85745f5-hb9cv" event={"ID":"339f5ec3-4c62-4174-96e1-dbc6a299f8de","Type":"ContainerDied","Data":"3e624310f87a90d602bb12fb6d477345867500d58d314403772c102c1a50e18b"} Sep 19 00:01:44.339231 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.339207 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:01:44.396411 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.396377 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6hpr\" (UniqueName: \"kubernetes.io/projected/339f5ec3-4c62-4174-96e1-dbc6a299f8de-kube-api-access-x6hpr\") pod \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " Sep 19 00:01:44.396590 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.396417 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-oauth-serving-cert\") pod \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " Sep 19 00:01:44.396590 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.396447 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-trusted-ca-bundle\") pod \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " Sep 19 00:01:44.396590 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.396469 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-service-ca\") pod \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " Sep 19 00:01:44.396844 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.396650 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-oauth-config\") pod \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " Sep 19 00:01:44.396844 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.396690 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-serving-cert\") pod \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " Sep 19 00:01:44.396844 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.396726 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-config\") pod \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\" (UID: \"339f5ec3-4c62-4174-96e1-dbc6a299f8de\") " Sep 19 00:01:44.396992 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.396837 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-service-ca" (OuterVolumeSpecName: "service-ca") pod "339f5ec3-4c62-4174-96e1-dbc6a299f8de" (UID: "339f5ec3-4c62-4174-96e1-dbc6a299f8de"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 00:01:44.396992 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.396852 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "339f5ec3-4c62-4174-96e1-dbc6a299f8de" (UID: "339f5ec3-4c62-4174-96e1-dbc6a299f8de"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 00:01:44.396992 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.396888 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "339f5ec3-4c62-4174-96e1-dbc6a299f8de" (UID: "339f5ec3-4c62-4174-96e1-dbc6a299f8de"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 00:01:44.397134 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.397095 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-oauth-serving-cert\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:44.397134 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.397119 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-trusted-ca-bundle\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:44.397203 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.397135 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-service-ca\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:44.397365 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.397345 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-config" (OuterVolumeSpecName: "console-config") pod "339f5ec3-4c62-4174-96e1-dbc6a299f8de" (UID: "339f5ec3-4c62-4174-96e1-dbc6a299f8de"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 00:01:44.398861 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.398829 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "339f5ec3-4c62-4174-96e1-dbc6a299f8de" (UID: "339f5ec3-4c62-4174-96e1-dbc6a299f8de"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 00:01:44.398960 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.398910 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/339f5ec3-4c62-4174-96e1-dbc6a299f8de-kube-api-access-x6hpr" (OuterVolumeSpecName: "kube-api-access-x6hpr") pod "339f5ec3-4c62-4174-96e1-dbc6a299f8de" (UID: "339f5ec3-4c62-4174-96e1-dbc6a299f8de"). InnerVolumeSpecName "kube-api-access-x6hpr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 00:01:44.398960 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.398916 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "339f5ec3-4c62-4174-96e1-dbc6a299f8de" (UID: "339f5ec3-4c62-4174-96e1-dbc6a299f8de"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 00:01:44.498341 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.498303 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6hpr\" (UniqueName: \"kubernetes.io/projected/339f5ec3-4c62-4174-96e1-dbc6a299f8de-kube-api-access-x6hpr\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:44.498341 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.498334 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-oauth-config\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:44.498341 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.498344 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-serving-cert\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:44.498554 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:44.498353 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/339f5ec3-4c62-4174-96e1-dbc6a299f8de-console-config\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:01:45.251712 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:45.251676 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85c85745f5-hb9cv" event={"ID":"339f5ec3-4c62-4174-96e1-dbc6a299f8de","Type":"ContainerDied","Data":"5e198dc879b005e4c2cd0fb4e5a9efa48ed696158db3fcc16fc7ae7244fbfc8c"} Sep 19 00:01:45.251712 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:45.251704 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85c85745f5-hb9cv" Sep 19 00:01:45.252243 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:45.251739 2568 scope.go:117] "RemoveContainer" containerID="3e624310f87a90d602bb12fb6d477345867500d58d314403772c102c1a50e18b" Sep 19 00:01:45.270688 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:45.270666 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85c85745f5-hb9cv"] Sep 19 00:01:45.274781 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:45.274760 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85c85745f5-hb9cv"] Sep 19 00:01:45.378892 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:01:45.378863 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="339f5ec3-4c62-4174-96e1-dbc6a299f8de" path="/var/lib/kubelet/pods/339f5ec3-4c62-4174-96e1-dbc6a299f8de/volumes" Sep 19 00:02:23.062914 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.062885 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d48994b45-f65t4"] Sep 19 00:02:23.063374 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.063153 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="339f5ec3-4c62-4174-96e1-dbc6a299f8de" containerName="console" Sep 19 00:02:23.063374 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.063164 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="339f5ec3-4c62-4174-96e1-dbc6a299f8de" containerName="console" Sep 19 00:02:23.063374 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.063206 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="339f5ec3-4c62-4174-96e1-dbc6a299f8de" containerName="console" Sep 19 00:02:23.065141 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.065120 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.067583 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.067558 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 19 00:02:23.068915 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.068894 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 19 00:02:23.069014 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.068910 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 19 00:02:23.069014 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.068915 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jsngw\"" Sep 19 00:02:23.069014 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.068976 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 19 00:02:23.069014 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.069004 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 19 00:02:23.073627 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.073609 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 19 00:02:23.076854 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.076832 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d48994b45-f65t4"] Sep 19 00:02:23.169471 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.169438 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-trusted-ca-bundle\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.169637 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.169490 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-console-config\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.169637 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.169531 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-service-ca\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.169637 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.169576 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-oauth-serving-cert\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.169637 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.169604 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e511b452-1df5-45a0-8a63-3361389068cb-console-serving-cert\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.169637 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.169622 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qmfz\" (UniqueName: \"kubernetes.io/projected/e511b452-1df5-45a0-8a63-3361389068cb-kube-api-access-7qmfz\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.169852 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.169675 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e511b452-1df5-45a0-8a63-3361389068cb-console-oauth-config\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.270753 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.270722 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e511b452-1df5-45a0-8a63-3361389068cb-console-serving-cert\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.270890 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.270758 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qmfz\" (UniqueName: \"kubernetes.io/projected/e511b452-1df5-45a0-8a63-3361389068cb-kube-api-access-7qmfz\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.270890 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.270776 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e511b452-1df5-45a0-8a63-3361389068cb-console-oauth-config\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.270890 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.270851 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-trusted-ca-bundle\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.271109 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.271087 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-console-config\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.271163 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.271125 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-service-ca\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.271215 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.271176 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-oauth-serving-cert\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.271759 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.271735 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-console-config\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.271946 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.271931 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-oauth-serving-cert\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.272055 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.272036 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-service-ca\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.272147 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.272127 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-trusted-ca-bundle\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.273398 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.273375 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e511b452-1df5-45a0-8a63-3361389068cb-console-serving-cert\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.273511 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.273474 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e511b452-1df5-45a0-8a63-3361389068cb-console-oauth-config\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.278418 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.278397 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qmfz\" (UniqueName: \"kubernetes.io/projected/e511b452-1df5-45a0-8a63-3361389068cb-kube-api-access-7qmfz\") pod \"console-d48994b45-f65t4\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.375681 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.375652 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:23.641907 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:23.641861 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d48994b45-f65t4"] Sep 19 00:02:24.388153 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:24.388116 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d48994b45-f65t4" event={"ID":"e511b452-1df5-45a0-8a63-3361389068cb","Type":"ContainerStarted","Data":"bd8fdf4170c402ac29f097f8b4648d15f087e7d251bb65efedc3f73db715eb50"} Sep 19 00:02:24.388533 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:24.388163 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d48994b45-f65t4" event={"ID":"e511b452-1df5-45a0-8a63-3361389068cb","Type":"ContainerStarted","Data":"bba04d506af656705dea868876f648bfd532d8097e32d7380757a0c97bf6bb28"} Sep 19 00:02:24.405141 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:24.405096 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d48994b45-f65t4" podStartSLOduration=1.405080126 podStartE2EDuration="1.405080126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:02:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:02:24.403466163 +0000 UTC m=+235.557060030" watchObservedRunningTime="2026-09-19 00:02:24.405080126 +0000 UTC m=+235.558673990" Sep 19 00:02:33.378697 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:33.378667 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:33.379138 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:33.378724 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:33.380741 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:33.380725 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:02:33.421907 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:02:33.421879 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:03:29.283114 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:29.283086 2568 kubelet.go:1648] "Image garbage collection succeeded" Sep 19 00:03:52.604344 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:52.604313 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-b75hz"] Sep 19 00:03:52.606271 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:52.606254 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-b75hz" Sep 19 00:03:52.608886 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:52.608864 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 19 00:03:52.608886 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:52.608864 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 19 00:03:52.609875 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:52.609854 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-b2rbt\"" Sep 19 00:03:52.617605 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:52.617582 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-b75hz"] Sep 19 00:03:52.719980 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:52.719952 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3a02266-8280-46ea-9576-882db0044ebe-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-b75hz\" (UID: \"f3a02266-8280-46ea-9576-882db0044ebe\") " pod="cert-manager/cert-manager-webhook-6486ffd997-b75hz" Sep 19 00:03:52.720145 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:52.720008 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26ntb\" (UniqueName: \"kubernetes.io/projected/f3a02266-8280-46ea-9576-882db0044ebe-kube-api-access-26ntb\") pod \"cert-manager-webhook-6486ffd997-b75hz\" (UID: \"f3a02266-8280-46ea-9576-882db0044ebe\") " pod="cert-manager/cert-manager-webhook-6486ffd997-b75hz" Sep 19 00:03:52.820903 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:52.820870 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3a02266-8280-46ea-9576-882db0044ebe-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-b75hz\" (UID: \"f3a02266-8280-46ea-9576-882db0044ebe\") " pod="cert-manager/cert-manager-webhook-6486ffd997-b75hz" Sep 19 00:03:52.821035 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:52.820920 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26ntb\" (UniqueName: \"kubernetes.io/projected/f3a02266-8280-46ea-9576-882db0044ebe-kube-api-access-26ntb\") pod \"cert-manager-webhook-6486ffd997-b75hz\" (UID: \"f3a02266-8280-46ea-9576-882db0044ebe\") " pod="cert-manager/cert-manager-webhook-6486ffd997-b75hz" Sep 19 00:03:52.829003 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:52.828977 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3a02266-8280-46ea-9576-882db0044ebe-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-b75hz\" (UID: \"f3a02266-8280-46ea-9576-882db0044ebe\") " pod="cert-manager/cert-manager-webhook-6486ffd997-b75hz" Sep 19 00:03:52.829125 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:52.829042 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26ntb\" (UniqueName: \"kubernetes.io/projected/f3a02266-8280-46ea-9576-882db0044ebe-kube-api-access-26ntb\") pod \"cert-manager-webhook-6486ffd997-b75hz\" (UID: \"f3a02266-8280-46ea-9576-882db0044ebe\") " pod="cert-manager/cert-manager-webhook-6486ffd997-b75hz" Sep 19 00:03:52.915036 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:52.915004 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-b75hz" Sep 19 00:03:53.181366 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:53.181290 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-b75hz"] Sep 19 00:03:53.184415 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:03:53.184391 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3a02266_8280_46ea_9576_882db0044ebe.slice/crio-82bacd8eba1c641078024a00f2a43261c87b039f3e56485cc59b899eae5381e9 WatchSource:0}: Error finding container 82bacd8eba1c641078024a00f2a43261c87b039f3e56485cc59b899eae5381e9: Status 404 returned error can't find the container with id 82bacd8eba1c641078024a00f2a43261c87b039f3e56485cc59b899eae5381e9 Sep 19 00:03:53.186775 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:53.186758 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 00:03:53.704949 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:53.704915 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-b75hz" event={"ID":"f3a02266-8280-46ea-9576-882db0044ebe","Type":"ContainerStarted","Data":"82bacd8eba1c641078024a00f2a43261c87b039f3e56485cc59b899eae5381e9"} Sep 19 00:03:59.729293 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:59.729253 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-b75hz" event={"ID":"f3a02266-8280-46ea-9576-882db0044ebe","Type":"ContainerStarted","Data":"4cedd66294d448a61ede66685a30e5605e624448e85469c9350722073ee6b184"} Sep 19 00:03:59.729772 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:59.729479 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-b75hz" Sep 19 00:03:59.745811 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:03:59.745754 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-b75hz" podStartSLOduration=2.228768131 podStartE2EDuration="7.745739288s" totalImagesPullingTime="5.516971157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:03:59.745286515 +0000 UTC m=+330.898880381" watchObservedRunningTime="2026-09-19 00:03:59.745739288 +0000 UTC m=+330.899333152" Sep 19 00:04:05.734875 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:05.734834 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-b75hz" Sep 19 00:04:23.714431 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.714351 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd"] Sep 19 00:04:23.716535 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.716512 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:23.719060 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.719034 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 19 00:04:23.719060 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.719051 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 19 00:04:23.719440 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.719399 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 19 00:04:23.719510 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.719447 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tz5mp\"" Sep 19 00:04:23.719844 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.719826 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 19 00:04:23.749086 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.749058 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd"] Sep 19 00:04:23.859410 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.859378 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6lgj\" (UniqueName: \"kubernetes.io/projected/68dc1819-e2f2-4c8d-b2a0-71ae6526698a-kube-api-access-f6lgj\") pod \"opendatahub-operator-controller-manager-577c99c85d-5qdtd\" (UID: \"68dc1819-e2f2-4c8d-b2a0-71ae6526698a\") " pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:23.859573 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.859421 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/68dc1819-e2f2-4c8d-b2a0-71ae6526698a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-577c99c85d-5qdtd\" (UID: \"68dc1819-e2f2-4c8d-b2a0-71ae6526698a\") " pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:23.859573 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.859454 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/68dc1819-e2f2-4c8d-b2a0-71ae6526698a-manifests\") pod \"opendatahub-operator-controller-manager-577c99c85d-5qdtd\" (UID: \"68dc1819-e2f2-4c8d-b2a0-71ae6526698a\") " pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:23.859573 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.859526 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68dc1819-e2f2-4c8d-b2a0-71ae6526698a-webhook-cert\") pod \"opendatahub-operator-controller-manager-577c99c85d-5qdtd\" (UID: \"68dc1819-e2f2-4c8d-b2a0-71ae6526698a\") " pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:23.859573 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.859568 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68dc1819-e2f2-4c8d-b2a0-71ae6526698a-tmp\") pod \"opendatahub-operator-controller-manager-577c99c85d-5qdtd\" (UID: \"68dc1819-e2f2-4c8d-b2a0-71ae6526698a\") " pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:23.960089 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.960043 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/68dc1819-e2f2-4c8d-b2a0-71ae6526698a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-577c99c85d-5qdtd\" (UID: \"68dc1819-e2f2-4c8d-b2a0-71ae6526698a\") " pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:23.960236 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.960106 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/68dc1819-e2f2-4c8d-b2a0-71ae6526698a-manifests\") pod \"opendatahub-operator-controller-manager-577c99c85d-5qdtd\" (UID: \"68dc1819-e2f2-4c8d-b2a0-71ae6526698a\") " pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:23.960236 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.960128 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68dc1819-e2f2-4c8d-b2a0-71ae6526698a-webhook-cert\") pod \"opendatahub-operator-controller-manager-577c99c85d-5qdtd\" (UID: \"68dc1819-e2f2-4c8d-b2a0-71ae6526698a\") " pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:23.960236 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.960154 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68dc1819-e2f2-4c8d-b2a0-71ae6526698a-tmp\") pod \"opendatahub-operator-controller-manager-577c99c85d-5qdtd\" (UID: \"68dc1819-e2f2-4c8d-b2a0-71ae6526698a\") " pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:23.960236 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.960199 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6lgj\" (UniqueName: \"kubernetes.io/projected/68dc1819-e2f2-4c8d-b2a0-71ae6526698a-kube-api-access-f6lgj\") pod \"opendatahub-operator-controller-manager-577c99c85d-5qdtd\" (UID: \"68dc1819-e2f2-4c8d-b2a0-71ae6526698a\") " pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:23.960530 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.960504 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/68dc1819-e2f2-4c8d-b2a0-71ae6526698a-manifests\") pod \"opendatahub-operator-controller-manager-577c99c85d-5qdtd\" (UID: \"68dc1819-e2f2-4c8d-b2a0-71ae6526698a\") " pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:23.962499 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.962473 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68dc1819-e2f2-4c8d-b2a0-71ae6526698a-tmp\") pod \"opendatahub-operator-controller-manager-577c99c85d-5qdtd\" (UID: \"68dc1819-e2f2-4c8d-b2a0-71ae6526698a\") " pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:23.962754 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.962738 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/68dc1819-e2f2-4c8d-b2a0-71ae6526698a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-577c99c85d-5qdtd\" (UID: \"68dc1819-e2f2-4c8d-b2a0-71ae6526698a\") " pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:23.962835 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.962813 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68dc1819-e2f2-4c8d-b2a0-71ae6526698a-webhook-cert\") pod \"opendatahub-operator-controller-manager-577c99c85d-5qdtd\" (UID: \"68dc1819-e2f2-4c8d-b2a0-71ae6526698a\") " pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:23.973653 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:23.973596 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6lgj\" (UniqueName: \"kubernetes.io/projected/68dc1819-e2f2-4c8d-b2a0-71ae6526698a-kube-api-access-f6lgj\") pod \"opendatahub-operator-controller-manager-577c99c85d-5qdtd\" (UID: \"68dc1819-e2f2-4c8d-b2a0-71ae6526698a\") " pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:24.026686 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:24.026653 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:24.307075 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:24.306992 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd"] Sep 19 00:04:24.310566 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:04:24.310534 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68dc1819_e2f2_4c8d_b2a0_71ae6526698a.slice/crio-39db9d36f62514e2de8fc44fddec252ace25972235f05ed611ae0ec6ea5ece3f WatchSource:0}: Error finding container 39db9d36f62514e2de8fc44fddec252ace25972235f05ed611ae0ec6ea5ece3f: Status 404 returned error can't find the container with id 39db9d36f62514e2de8fc44fddec252ace25972235f05ed611ae0ec6ea5ece3f Sep 19 00:04:24.817585 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:24.817548 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" event={"ID":"68dc1819-e2f2-4c8d-b2a0-71ae6526698a","Type":"ContainerStarted","Data":"39db9d36f62514e2de8fc44fddec252ace25972235f05ed611ae0ec6ea5ece3f"} Sep 19 00:04:27.832575 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:27.832536 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" event={"ID":"68dc1819-e2f2-4c8d-b2a0-71ae6526698a","Type":"ContainerStarted","Data":"c5029ced9a9e881152f2165d7ffc6dbd62e93a10f4f20edb14508c84445548a4"} Sep 19 00:04:28.838581 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:28.838550 2568 generic.go:356] "Generic (PLEG): container finished" podID="68dc1819-e2f2-4c8d-b2a0-71ae6526698a" containerID="c5029ced9a9e881152f2165d7ffc6dbd62e93a10f4f20edb14508c84445548a4" exitCode=0 Sep 19 00:04:28.838975 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:28.838610 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" event={"ID":"68dc1819-e2f2-4c8d-b2a0-71ae6526698a","Type":"ContainerDied","Data":"c5029ced9a9e881152f2165d7ffc6dbd62e93a10f4f20edb14508c84445548a4"} Sep 19 00:04:30.847402 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:30.847368 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" event={"ID":"68dc1819-e2f2-4c8d-b2a0-71ae6526698a","Type":"ContainerStarted","Data":"6a98be8495df7a7f85cf331e7e8032ab5b9971b74de157c303223b8f3604bddd"} Sep 19 00:04:30.847769 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:30.847497 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:30.868503 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:30.868451 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" podStartSLOduration=2.895050286 podStartE2EDuration="7.868435917s" totalImagesPullingTime="3.91047474s" totalInitContainerRuntime="1.062910891s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:04:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:04:30.866249204 +0000 UTC m=+362.019843069" watchObservedRunningTime="2026-09-19 00:04:30.868435917 +0000 UTC m=+362.022029829" Sep 19 00:04:41.853154 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:41.853123 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-577c99c85d-5qdtd" Sep 19 00:04:49.033413 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.033384 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm"] Sep 19 00:04:49.039150 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.039129 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm" Sep 19 00:04:49.041573 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.041548 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 19 00:04:49.042573 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.042537 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-rrz4m\"" Sep 19 00:04:49.042573 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.042537 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 19 00:04:49.042736 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.042582 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 19 00:04:49.042736 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.042583 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 19 00:04:49.049703 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.049682 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm"] Sep 19 00:04:49.062649 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.062625 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m22hk\" (UniqueName: \"kubernetes.io/projected/fcfe7377-8fff-48f2-a4ec-d111f474103f-kube-api-access-m22hk\") pod \"kube-auth-proxy-755c79bd44-h9hhm\" (UID: \"fcfe7377-8fff-48f2-a4ec-d111f474103f\") " pod="openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm" Sep 19 00:04:49.062746 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.062675 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcfe7377-8fff-48f2-a4ec-d111f474103f-tmp\") pod \"kube-auth-proxy-755c79bd44-h9hhm\" (UID: \"fcfe7377-8fff-48f2-a4ec-d111f474103f\") " pod="openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm" Sep 19 00:04:49.062746 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.062724 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcfe7377-8fff-48f2-a4ec-d111f474103f-tls-certs\") pod \"kube-auth-proxy-755c79bd44-h9hhm\" (UID: \"fcfe7377-8fff-48f2-a4ec-d111f474103f\") " pod="openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm" Sep 19 00:04:49.163981 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.163954 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m22hk\" (UniqueName: \"kubernetes.io/projected/fcfe7377-8fff-48f2-a4ec-d111f474103f-kube-api-access-m22hk\") pod \"kube-auth-proxy-755c79bd44-h9hhm\" (UID: \"fcfe7377-8fff-48f2-a4ec-d111f474103f\") " pod="openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm" Sep 19 00:04:49.164143 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.164019 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcfe7377-8fff-48f2-a4ec-d111f474103f-tmp\") pod \"kube-auth-proxy-755c79bd44-h9hhm\" (UID: \"fcfe7377-8fff-48f2-a4ec-d111f474103f\") " pod="openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm" Sep 19 00:04:49.164143 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.164052 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcfe7377-8fff-48f2-a4ec-d111f474103f-tls-certs\") pod \"kube-auth-proxy-755c79bd44-h9hhm\" (UID: \"fcfe7377-8fff-48f2-a4ec-d111f474103f\") " pod="openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm" Sep 19 00:04:49.164218 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:04:49.164178 2568 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Sep 19 00:04:49.164251 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:04:49.164236 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcfe7377-8fff-48f2-a4ec-d111f474103f-tls-certs podName:fcfe7377-8fff-48f2-a4ec-d111f474103f nodeName:}" failed. No retries permitted until 2026-09-19 00:04:49.664218097 +0000 UTC m=+380.817811944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/fcfe7377-8fff-48f2-a4ec-d111f474103f-tls-certs") pod "kube-auth-proxy-755c79bd44-h9hhm" (UID: "fcfe7377-8fff-48f2-a4ec-d111f474103f") : secret "kube-auth-proxy-tls" not found Sep 19 00:04:49.166245 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.166221 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcfe7377-8fff-48f2-a4ec-d111f474103f-tmp\") pod \"kube-auth-proxy-755c79bd44-h9hhm\" (UID: \"fcfe7377-8fff-48f2-a4ec-d111f474103f\") " pod="openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm" Sep 19 00:04:49.180607 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.180577 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m22hk\" (UniqueName: \"kubernetes.io/projected/fcfe7377-8fff-48f2-a4ec-d111f474103f-kube-api-access-m22hk\") pod \"kube-auth-proxy-755c79bd44-h9hhm\" (UID: \"fcfe7377-8fff-48f2-a4ec-d111f474103f\") " pod="openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm" Sep 19 00:04:49.667694 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.667658 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcfe7377-8fff-48f2-a4ec-d111f474103f-tls-certs\") pod \"kube-auth-proxy-755c79bd44-h9hhm\" (UID: \"fcfe7377-8fff-48f2-a4ec-d111f474103f\") " pod="openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm" Sep 19 00:04:49.670211 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.670181 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcfe7377-8fff-48f2-a4ec-d111f474103f-tls-certs\") pod \"kube-auth-proxy-755c79bd44-h9hhm\" (UID: \"fcfe7377-8fff-48f2-a4ec-d111f474103f\") " pod="openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm" Sep 19 00:04:49.948705 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:49.948624 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm" Sep 19 00:04:50.226735 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:50.226663 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm"] Sep 19 00:04:50.234073 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:04:50.234028 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcfe7377_8fff_48f2_a4ec_d111f474103f.slice/crio-1c0f104cab9c146d50c3725e1908fe5dc7c7127a4b64c644b95c477a65baed53 WatchSource:0}: Error finding container 1c0f104cab9c146d50c3725e1908fe5dc7c7127a4b64c644b95c477a65baed53: Status 404 returned error can't find the container with id 1c0f104cab9c146d50c3725e1908fe5dc7c7127a4b64c644b95c477a65baed53 Sep 19 00:04:50.920284 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:50.920245 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm" event={"ID":"fcfe7377-8fff-48f2-a4ec-d111f474103f","Type":"ContainerStarted","Data":"1c0f104cab9c146d50c3725e1908fe5dc7c7127a4b64c644b95c477a65baed53"} Sep 19 00:04:53.937572 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:53.937540 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm" event={"ID":"fcfe7377-8fff-48f2-a4ec-d111f474103f","Type":"ContainerStarted","Data":"3479bdf525205d75cc28a4a1bc99bce97b09deec45b8032183cb2a1f0727d299"} Sep 19 00:04:53.952696 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:53.952641 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-755c79bd44-h9hhm" podStartSLOduration=2.010573023 podStartE2EDuration="4.952624507s" totalImagesPullingTime="2.942051484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:04:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:04:53.951509677 +0000 UTC m=+385.105103542" watchObservedRunningTime="2026-09-19 00:04:53.952624507 +0000 UTC m=+385.106218371" Sep 19 00:04:54.520744 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:54.520708 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kwnhz"] Sep 19 00:04:54.524012 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:54.523997 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kwnhz" Sep 19 00:04:54.526297 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:54.526272 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 19 00:04:54.526421 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:54.526390 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-v6fk6\"" Sep 19 00:04:54.531857 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:54.531833 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kwnhz"] Sep 19 00:04:54.606084 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:54.606051 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcqzq\" (UniqueName: \"kubernetes.io/projected/f2735ad3-d056-44c9-b275-9b7c3b940294-kube-api-access-qcqzq\") pod \"odh-model-controller-7998c67dc4-kwnhz\" (UID: \"f2735ad3-d056-44c9-b275-9b7c3b940294\") " pod="opendatahub/odh-model-controller-7998c67dc4-kwnhz" Sep 19 00:04:54.606283 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:54.606111 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2735ad3-d056-44c9-b275-9b7c3b940294-cert\") pod \"odh-model-controller-7998c67dc4-kwnhz\" (UID: \"f2735ad3-d056-44c9-b275-9b7c3b940294\") " pod="opendatahub/odh-model-controller-7998c67dc4-kwnhz" Sep 19 00:04:54.707405 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:54.707365 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2735ad3-d056-44c9-b275-9b7c3b940294-cert\") pod \"odh-model-controller-7998c67dc4-kwnhz\" (UID: \"f2735ad3-d056-44c9-b275-9b7c3b940294\") " pod="opendatahub/odh-model-controller-7998c67dc4-kwnhz" Sep 19 00:04:54.707598 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:54.707435 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcqzq\" (UniqueName: \"kubernetes.io/projected/f2735ad3-d056-44c9-b275-9b7c3b940294-kube-api-access-qcqzq\") pod \"odh-model-controller-7998c67dc4-kwnhz\" (UID: \"f2735ad3-d056-44c9-b275-9b7c3b940294\") " pod="opendatahub/odh-model-controller-7998c67dc4-kwnhz" Sep 19 00:04:54.707598 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:04:54.707539 2568 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 19 00:04:54.707713 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:04:54.707625 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2735ad3-d056-44c9-b275-9b7c3b940294-cert podName:f2735ad3-d056-44c9-b275-9b7c3b940294 nodeName:}" failed. No retries permitted until 2026-09-19 00:04:55.207603106 +0000 UTC m=+386.361196952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f2735ad3-d056-44c9-b275-9b7c3b940294-cert") pod "odh-model-controller-7998c67dc4-kwnhz" (UID: "f2735ad3-d056-44c9-b275-9b7c3b940294") : secret "odh-model-controller-webhook-cert" not found Sep 19 00:04:54.715223 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:54.715200 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcqzq\" (UniqueName: \"kubernetes.io/projected/f2735ad3-d056-44c9-b275-9b7c3b940294-kube-api-access-qcqzq\") pod \"odh-model-controller-7998c67dc4-kwnhz\" (UID: \"f2735ad3-d056-44c9-b275-9b7c3b940294\") " pod="opendatahub/odh-model-controller-7998c67dc4-kwnhz" Sep 19 00:04:55.118329 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.118296 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ksrzk"] Sep 19 00:04:55.121785 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.121760 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-ksrzk" Sep 19 00:04:55.124076 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.124052 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-pp5gc\"" Sep 19 00:04:55.124236 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.124213 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 19 00:04:55.129873 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.129849 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb"] Sep 19 00:04:55.133113 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.133083 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.134993 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.134973 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ksrzk"] Sep 19 00:04:55.135475 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.135454 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-lmfcp\"" Sep 19 00:04:55.135577 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.135506 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 19 00:04:55.145185 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.145160 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb"] Sep 19 00:04:55.212553 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.212516 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6582\" (UniqueName: \"kubernetes.io/projected/68e945ce-02ea-4a56-b338-7c769d1e3553-kube-api-access-q6582\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.212553 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.212553 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.212818 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.212581 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whbcs\" (UniqueName: \"kubernetes.io/projected/2edefa5f-0032-4bc8-a0ad-0fd54ef16e55-kube-api-access-whbcs\") pod \"model-serving-api-54d89bd5fc-ksrzk\" (UID: \"2edefa5f-0032-4bc8-a0ad-0fd54ef16e55\") " pod="opendatahub/model-serving-api-54d89bd5fc-ksrzk" Sep 19 00:04:55.212818 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.212657 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68e945ce-02ea-4a56-b338-7c769d1e3553-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.212818 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.212687 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.212818 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.212706 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.212818 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.212722 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.212818 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.212762 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-data\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.212818 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.212786 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.213113 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.212841 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2735ad3-d056-44c9-b275-9b7c3b940294-cert\") pod \"odh-model-controller-7998c67dc4-kwnhz\" (UID: \"f2735ad3-d056-44c9-b275-9b7c3b940294\") " pod="opendatahub/odh-model-controller-7998c67dc4-kwnhz" Sep 19 00:04:55.213113 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.212881 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2edefa5f-0032-4bc8-a0ad-0fd54ef16e55-tls-certs\") pod \"model-serving-api-54d89bd5fc-ksrzk\" (UID: \"2edefa5f-0032-4bc8-a0ad-0fd54ef16e55\") " pod="opendatahub/model-serving-api-54d89bd5fc-ksrzk" Sep 19 00:04:55.213113 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.212914 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-token\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.215271 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.215244 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f2735ad3-d056-44c9-b275-9b7c3b940294-cert\") pod \"odh-model-controller-7998c67dc4-kwnhz\" (UID: \"f2735ad3-d056-44c9-b275-9b7c3b940294\") " pod="opendatahub/odh-model-controller-7998c67dc4-kwnhz" Sep 19 00:04:55.314130 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.314090 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.314130 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.314133 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.314385 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.314149 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.314385 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.314275 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-data\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.314385 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.314328 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.314539 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.314385 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2edefa5f-0032-4bc8-a0ad-0fd54ef16e55-tls-certs\") pod \"model-serving-api-54d89bd5fc-ksrzk\" (UID: \"2edefa5f-0032-4bc8-a0ad-0fd54ef16e55\") " pod="opendatahub/model-serving-api-54d89bd5fc-ksrzk" Sep 19 00:04:55.314539 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.314425 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-token\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.314539 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.314454 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6582\" (UniqueName: \"kubernetes.io/projected/68e945ce-02ea-4a56-b338-7c769d1e3553-kube-api-access-q6582\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.314539 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.314486 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.314539 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.314510 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.314539 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.314529 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whbcs\" (UniqueName: \"kubernetes.io/projected/2edefa5f-0032-4bc8-a0ad-0fd54ef16e55-kube-api-access-whbcs\") pod \"model-serving-api-54d89bd5fc-ksrzk\" (UID: \"2edefa5f-0032-4bc8-a0ad-0fd54ef16e55\") " pod="opendatahub/model-serving-api-54d89bd5fc-ksrzk" Sep 19 00:04:55.314860 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.314579 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68e945ce-02ea-4a56-b338-7c769d1e3553-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.314860 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.314621 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-data\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.314860 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.314677 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.315201 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.314895 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.315284 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.315263 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68e945ce-02ea-4a56-b338-7c769d1e3553-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.316729 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.316700 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.316889 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.316846 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.317061 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.317041 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2edefa5f-0032-4bc8-a0ad-0fd54ef16e55-tls-certs\") pod \"model-serving-api-54d89bd5fc-ksrzk\" (UID: \"2edefa5f-0032-4bc8-a0ad-0fd54ef16e55\") " pod="opendatahub/model-serving-api-54d89bd5fc-ksrzk" Sep 19 00:04:55.322096 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.322072 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whbcs\" (UniqueName: \"kubernetes.io/projected/2edefa5f-0032-4bc8-a0ad-0fd54ef16e55-kube-api-access-whbcs\") pod \"model-serving-api-54d89bd5fc-ksrzk\" (UID: \"2edefa5f-0032-4bc8-a0ad-0fd54ef16e55\") " pod="opendatahub/model-serving-api-54d89bd5fc-ksrzk" Sep 19 00:04:55.322272 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.322254 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-token\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.322490 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.322470 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6582\" (UniqueName: \"kubernetes.io/projected/68e945ce-02ea-4a56-b338-7c769d1e3553-kube-api-access-q6582\") pod \"data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.435474 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.435392 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kwnhz" Sep 19 00:04:55.435614 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.435531 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-ksrzk" Sep 19 00:04:55.445859 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.445836 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:04:55.522329 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.522300 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq"] Sep 19 00:04:55.529038 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.529020 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.539649 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.539619 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq"] Sep 19 00:04:55.617584 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.617551 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1eed8ec-0fec-4500-aec4-788b93b81da7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.617733 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.617590 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1eed8ec-0fec-4500-aec4-788b93b81da7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.617733 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.617609 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1eed8ec-0fec-4500-aec4-788b93b81da7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.617733 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.617684 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1eed8ec-0fec-4500-aec4-788b93b81da7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.617733 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.617721 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1eed8ec-0fec-4500-aec4-788b93b81da7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.617918 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.617744 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1eed8ec-0fec-4500-aec4-788b93b81da7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.617918 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.617839 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2mr2\" (UniqueName: \"kubernetes.io/projected/b1eed8ec-0fec-4500-aec4-788b93b81da7-kube-api-access-f2mr2\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.617918 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.617858 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1eed8ec-0fec-4500-aec4-788b93b81da7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.617918 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.617873 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1eed8ec-0fec-4500-aec4-788b93b81da7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.715530 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.715498 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kwnhz"] Sep 19 00:04:55.718411 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.718378 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1eed8ec-0fec-4500-aec4-788b93b81da7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.718553 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.718432 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1eed8ec-0fec-4500-aec4-788b93b81da7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.718553 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:04:55.718385 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2735ad3_d056_44c9_b275_9b7c3b940294.slice/crio-1725549691f25f5473dfae80cfe2599524b1835fff6f0d43f8617e319e1dd6ec WatchSource:0}: Error finding container 1725549691f25f5473dfae80cfe2599524b1835fff6f0d43f8617e319e1dd6ec: Status 404 returned error can't find the container with id 1725549691f25f5473dfae80cfe2599524b1835fff6f0d43f8617e319e1dd6ec Sep 19 00:04:55.718553 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.718496 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2mr2\" (UniqueName: \"kubernetes.io/projected/b1eed8ec-0fec-4500-aec4-788b93b81da7-kube-api-access-f2mr2\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.718553 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.718525 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1eed8ec-0fec-4500-aec4-788b93b81da7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.718553 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.718551 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1eed8ec-0fec-4500-aec4-788b93b81da7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.718880 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.718603 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1eed8ec-0fec-4500-aec4-788b93b81da7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.718880 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.718639 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1eed8ec-0fec-4500-aec4-788b93b81da7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.718880 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.718665 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1eed8ec-0fec-4500-aec4-788b93b81da7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.718880 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.718708 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1eed8ec-0fec-4500-aec4-788b93b81da7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.719172 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.719085 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ksrzk"] Sep 19 00:04:55.719172 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.719100 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1eed8ec-0fec-4500-aec4-788b93b81da7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.719453 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.719386 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1eed8ec-0fec-4500-aec4-788b93b81da7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.719514 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.719462 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1eed8ec-0fec-4500-aec4-788b93b81da7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.719514 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.719506 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1eed8ec-0fec-4500-aec4-788b93b81da7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.720245 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.719947 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1eed8ec-0fec-4500-aec4-788b93b81da7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.721584 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.721543 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1eed8ec-0fec-4500-aec4-788b93b81da7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.722711 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.722688 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1eed8ec-0fec-4500-aec4-788b93b81da7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.723172 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.723153 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb"] Sep 19 00:04:55.723373 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:04:55.723311 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2edefa5f_0032_4bc8_a0ad_0fd54ef16e55.slice/crio-1df27e35587c8f03ead8c67ddc1cbd1870c65190ea39010600f9893f10e2fdb6 WatchSource:0}: Error finding container 1df27e35587c8f03ead8c67ddc1cbd1870c65190ea39010600f9893f10e2fdb6: Status 404 returned error can't find the container with id 1df27e35587c8f03ead8c67ddc1cbd1870c65190ea39010600f9893f10e2fdb6 Sep 19 00:04:55.725742 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:04:55.725686 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68e945ce_02ea_4a56_b338_7c769d1e3553.slice/crio-c0715de20ab69e727f5f79bff2d8fb6bdb6065c6a5b5fd0cf711348d38cac607 WatchSource:0}: Error finding container c0715de20ab69e727f5f79bff2d8fb6bdb6065c6a5b5fd0cf711348d38cac607: Status 404 returned error can't find the container with id c0715de20ab69e727f5f79bff2d8fb6bdb6065c6a5b5fd0cf711348d38cac607 Sep 19 00:04:55.726480 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.726412 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1eed8ec-0fec-4500-aec4-788b93b81da7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.726774 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.726751 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2mr2\" (UniqueName: \"kubernetes.io/projected/b1eed8ec-0fec-4500-aec4-788b93b81da7-kube-api-access-f2mr2\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq79tq\" (UID: \"b1eed8ec-0fec-4500-aec4-788b93b81da7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.841056 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.841017 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:04:55.945777 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.945734 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" event={"ID":"68e945ce-02ea-4a56-b338-7c769d1e3553","Type":"ContainerStarted","Data":"c0715de20ab69e727f5f79bff2d8fb6bdb6065c6a5b5fd0cf711348d38cac607"} Sep 19 00:04:55.946965 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.946936 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kwnhz" event={"ID":"f2735ad3-d056-44c9-b275-9b7c3b940294","Type":"ContainerStarted","Data":"1725549691f25f5473dfae80cfe2599524b1835fff6f0d43f8617e319e1dd6ec"} Sep 19 00:04:55.947999 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:55.947979 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-ksrzk" event={"ID":"2edefa5f-0032-4bc8-a0ad-0fd54ef16e55","Type":"ContainerStarted","Data":"1df27e35587c8f03ead8c67ddc1cbd1870c65190ea39010600f9893f10e2fdb6"} Sep 19 00:04:56.125975 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:56.125951 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq"] Sep 19 00:04:56.127622 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:04:56.127588 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1eed8ec_0fec_4500_aec4_788b93b81da7.slice/crio-2572c932924b33fc5429db3ae989cb4663b500e6fe8398125b2eaca076818d30 WatchSource:0}: Error finding container 2572c932924b33fc5429db3ae989cb4663b500e6fe8398125b2eaca076818d30: Status 404 returned error can't find the container with id 2572c932924b33fc5429db3ae989cb4663b500e6fe8398125b2eaca076818d30 Sep 19 00:04:56.956987 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:04:56.956948 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" event={"ID":"b1eed8ec-0fec-4500-aec4-788b93b81da7","Type":"ContainerStarted","Data":"2572c932924b33fc5429db3ae989cb4663b500e6fe8398125b2eaca076818d30"} Sep 19 00:05:00.042742 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.042697 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 19 00:05:00.043077 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.042844 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 19 00:05:00.043077 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.042894 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 19 00:05:00.607865 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.607827 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc"] Sep 19 00:05:00.611115 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.611093 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" Sep 19 00:05:00.617173 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.617118 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 19 00:05:00.617173 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.617133 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 19 00:05:00.617663 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.617642 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lvwnq\"" Sep 19 00:05:00.617784 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.617678 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 19 00:05:00.617784 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.617697 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 19 00:05:00.617784 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.617683 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 19 00:05:00.634976 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.634947 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc"] Sep 19 00:05:00.660396 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.660362 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtfpl\" (UniqueName: \"kubernetes.io/projected/6237e094-18a6-4593-8525-94a7614f0d22-kube-api-access-xtfpl\") pod \"lws-controller-manager-7968c85699-z2mnc\" (UID: \"6237e094-18a6-4593-8525-94a7614f0d22\") " pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" Sep 19 00:05:00.660396 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.660399 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6237e094-18a6-4593-8525-94a7614f0d22-metrics-cert\") pod \"lws-controller-manager-7968c85699-z2mnc\" (UID: \"6237e094-18a6-4593-8525-94a7614f0d22\") " pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" Sep 19 00:05:00.660570 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.660420 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6237e094-18a6-4593-8525-94a7614f0d22-cert\") pod \"lws-controller-manager-7968c85699-z2mnc\" (UID: \"6237e094-18a6-4593-8525-94a7614f0d22\") " pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" Sep 19 00:05:00.660570 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.660535 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6237e094-18a6-4593-8525-94a7614f0d22-manager-config\") pod \"lws-controller-manager-7968c85699-z2mnc\" (UID: \"6237e094-18a6-4593-8525-94a7614f0d22\") " pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" Sep 19 00:05:00.761701 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.761660 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6237e094-18a6-4593-8525-94a7614f0d22-metrics-cert\") pod \"lws-controller-manager-7968c85699-z2mnc\" (UID: \"6237e094-18a6-4593-8525-94a7614f0d22\") " pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" Sep 19 00:05:00.761701 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.761705 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6237e094-18a6-4593-8525-94a7614f0d22-cert\") pod \"lws-controller-manager-7968c85699-z2mnc\" (UID: \"6237e094-18a6-4593-8525-94a7614f0d22\") " pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" Sep 19 00:05:00.761977 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.761768 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6237e094-18a6-4593-8525-94a7614f0d22-manager-config\") pod \"lws-controller-manager-7968c85699-z2mnc\" (UID: \"6237e094-18a6-4593-8525-94a7614f0d22\") " pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" Sep 19 00:05:00.761977 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.761879 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtfpl\" (UniqueName: \"kubernetes.io/projected/6237e094-18a6-4593-8525-94a7614f0d22-kube-api-access-xtfpl\") pod \"lws-controller-manager-7968c85699-z2mnc\" (UID: \"6237e094-18a6-4593-8525-94a7614f0d22\") " pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" Sep 19 00:05:00.762492 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.762471 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6237e094-18a6-4593-8525-94a7614f0d22-manager-config\") pod \"lws-controller-manager-7968c85699-z2mnc\" (UID: \"6237e094-18a6-4593-8525-94a7614f0d22\") " pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" Sep 19 00:05:00.764330 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.764313 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6237e094-18a6-4593-8525-94a7614f0d22-metrics-cert\") pod \"lws-controller-manager-7968c85699-z2mnc\" (UID: \"6237e094-18a6-4593-8525-94a7614f0d22\") " pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" Sep 19 00:05:00.764456 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.764439 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6237e094-18a6-4593-8525-94a7614f0d22-cert\") pod \"lws-controller-manager-7968c85699-z2mnc\" (UID: \"6237e094-18a6-4593-8525-94a7614f0d22\") " pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" Sep 19 00:05:00.769834 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.769787 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtfpl\" (UniqueName: \"kubernetes.io/projected/6237e094-18a6-4593-8525-94a7614f0d22-kube-api-access-xtfpl\") pod \"lws-controller-manager-7968c85699-z2mnc\" (UID: \"6237e094-18a6-4593-8525-94a7614f0d22\") " pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" Sep 19 00:05:00.920038 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.919962 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" Sep 19 00:05:00.981157 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.981117 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kwnhz" event={"ID":"f2735ad3-d056-44c9-b275-9b7c3b940294","Type":"ContainerStarted","Data":"9cf3f8b0bb339188952d510aaa128192115dea504c9f129895e7122a67a190bd"} Sep 19 00:05:00.981577 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.981369 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-kwnhz" Sep 19 00:05:00.983949 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.983346 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" event={"ID":"b1eed8ec-0fec-4500-aec4-788b93b81da7","Type":"ContainerStarted","Data":"2422c78fac2300d777cc64f8a990c8c1fb8cece43745aa9ec9534ae852fe06ed"} Sep 19 00:05:00.985666 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.985631 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-ksrzk" event={"ID":"2edefa5f-0032-4bc8-a0ad-0fd54ef16e55","Type":"ContainerStarted","Data":"0fd2e90ef405fac25e11ac895ba64f18af06279a706d4ba0d1214f525e040da1"} Sep 19 00:05:00.986188 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:00.986145 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-ksrzk" Sep 19 00:05:01.023241 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:01.023184 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-kwnhz" podStartSLOduration=2.700316305 podStartE2EDuration="7.023169524s" totalImagesPullingTime="4.322853219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:05:01.021644363 +0000 UTC m=+392.175238221" watchObservedRunningTime="2026-09-19 00:05:01.023169524 +0000 UTC m=+392.176763396" Sep 19 00:05:01.060697 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:01.060644 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" podStartSLOduration=2.14814489 podStartE2EDuration="6.060628188s" totalImagesPullingTime="3.912483298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:04:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:05:01.05981206 +0000 UTC m=+392.213405915" watchObservedRunningTime="2026-09-19 00:05:01.060628188 +0000 UTC m=+392.214222052" Sep 19 00:05:01.089783 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:01.089731 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-ksrzk" podStartSLOduration=1.783592777 podStartE2EDuration="6.089714677s" totalImagesPullingTime="4.3061219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:04:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:05:01.088380499 +0000 UTC m=+392.241974367" watchObservedRunningTime="2026-09-19 00:05:01.089714677 +0000 UTC m=+392.243308542" Sep 19 00:05:01.188679 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:01.188605 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc"] Sep 19 00:05:01.191396 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:05:01.191368 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6237e094_18a6_4593_8525_94a7614f0d22.slice/crio-61421c60c09934ef9eb3740e80fc6a9c98812e6453a0a8274588ba41ccd2c388 WatchSource:0}: Error finding container 61421c60c09934ef9eb3740e80fc6a9c98812e6453a0a8274588ba41ccd2c388: Status 404 returned error can't find the container with id 61421c60c09934ef9eb3740e80fc6a9c98812e6453a0a8274588ba41ccd2c388 Sep 19 00:05:01.841435 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:01.841404 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:05:01.846197 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:01.846161 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:05:01.991347 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:01.991305 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" event={"ID":"6237e094-18a6-4593-8525-94a7614f0d22","Type":"ContainerStarted","Data":"61421c60c09934ef9eb3740e80fc6a9c98812e6453a0a8274588ba41ccd2c388"} Sep 19 00:05:01.991771 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:01.991751 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:05:01.992438 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:01.992419 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq79tq" Sep 19 00:05:02.042610 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:02.042578 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb"] Sep 19 00:05:02.417422 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:02.416901 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 19 00:05:02.417422 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:02.417024 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 19 00:05:02.417422 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:02.417064 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"} Sep 19 00:05:02.997935 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:02.997892 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" event={"ID":"68e945ce-02ea-4a56-b338-7c769d1e3553","Type":"ContainerStarted","Data":"6d9e4db8a987b9cddbf7609dadaf1511d90e11c5c2dea0aa7e81a577950db5a8"} Sep 19 00:05:02.998335 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:02.998306 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" podUID="68e945ce-02ea-4a56-b338-7c769d1e3553" containerName="istio-proxy" containerID="cri-o://6d9e4db8a987b9cddbf7609dadaf1511d90e11c5c2dea0aa7e81a577950db5a8" gracePeriod=30 Sep 19 00:05:03.030498 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:03.030446 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" podStartSLOduration=1.342099317 podStartE2EDuration="8.030427145s" totalImagesPullingTime="6.688327828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:04:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:05:03.029468256 +0000 UTC m=+394.183062150" watchObservedRunningTime="2026-09-19 00:05:03.030427145 +0000 UTC m=+394.184020988" Sep 19 00:05:05.008291 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:05.008258 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" event={"ID":"6237e094-18a6-4593-8525-94a7614f0d22","Type":"ContainerStarted","Data":"c57a68a98332bdb74294c766de10fbb174a90f80f1548b3e0778811d5f12b1a5"} Sep 19 00:05:05.008698 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:05.008496 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" Sep 19 00:05:05.029659 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:05.029612 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" podStartSLOduration=2.160958244 podStartE2EDuration="5.029596418s" totalImagesPullingTime="2.868638174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:05:05.0272993 +0000 UTC m=+396.180893176" watchObservedRunningTime="2026-09-19 00:05:05.029596418 +0000 UTC m=+396.183190283" Sep 19 00:05:08.245975 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.245940 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:05:08.328666 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.328575 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6582\" (UniqueName: \"kubernetes.io/projected/68e945ce-02ea-4a56-b338-7c769d1e3553-kube-api-access-q6582\") pod \"68e945ce-02ea-4a56-b338-7c769d1e3553\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " Sep 19 00:05:08.328666 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.328622 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-envoy\") pod \"68e945ce-02ea-4a56-b338-7c769d1e3553\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " Sep 19 00:05:08.328921 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.328679 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-data\") pod \"68e945ce-02ea-4a56-b338-7c769d1e3553\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " Sep 19 00:05:08.328921 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.328702 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-workload-certs\") pod \"68e945ce-02ea-4a56-b338-7c769d1e3553\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " Sep 19 00:05:08.328921 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.328772 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-token\") pod \"68e945ce-02ea-4a56-b338-7c769d1e3553\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " Sep 19 00:05:08.328921 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.328842 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-credential-socket\") pod \"68e945ce-02ea-4a56-b338-7c769d1e3553\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " Sep 19 00:05:08.328921 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.328870 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68e945ce-02ea-4a56-b338-7c769d1e3553-istiod-ca-cert\") pod \"68e945ce-02ea-4a56-b338-7c769d1e3553\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " Sep 19 00:05:08.328921 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.328903 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-podinfo\") pod \"68e945ce-02ea-4a56-b338-7c769d1e3553\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " Sep 19 00:05:08.329258 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.328934 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-workload-socket\") pod \"68e945ce-02ea-4a56-b338-7c769d1e3553\" (UID: \"68e945ce-02ea-4a56-b338-7c769d1e3553\") " Sep 19 00:05:08.329258 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.328958 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-data" (OuterVolumeSpecName: "istio-data") pod "68e945ce-02ea-4a56-b338-7c769d1e3553" (UID: "68e945ce-02ea-4a56-b338-7c769d1e3553"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 00:05:08.329258 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.329100 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "68e945ce-02ea-4a56-b338-7c769d1e3553" (UID: "68e945ce-02ea-4a56-b338-7c769d1e3553"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 00:05:08.329258 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.329135 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "68e945ce-02ea-4a56-b338-7c769d1e3553" (UID: "68e945ce-02ea-4a56-b338-7c769d1e3553"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 00:05:08.329258 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.329229 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-data\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:05:08.329258 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.329256 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-workload-certs\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:05:08.329500 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.329259 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68e945ce-02ea-4a56-b338-7c769d1e3553-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "68e945ce-02ea-4a56-b338-7c769d1e3553" (UID: "68e945ce-02ea-4a56-b338-7c769d1e3553"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 00:05:08.329500 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.329273 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-credential-socket\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:05:08.329500 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.329301 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "68e945ce-02ea-4a56-b338-7c769d1e3553" (UID: "68e945ce-02ea-4a56-b338-7c769d1e3553"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 00:05:08.331109 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.331080 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "68e945ce-02ea-4a56-b338-7c769d1e3553" (UID: "68e945ce-02ea-4a56-b338-7c769d1e3553"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 00:05:08.331241 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.331202 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-token" (OuterVolumeSpecName: "istio-token") pod "68e945ce-02ea-4a56-b338-7c769d1e3553" (UID: "68e945ce-02ea-4a56-b338-7c769d1e3553"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 00:05:08.331241 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.331223 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68e945ce-02ea-4a56-b338-7c769d1e3553-kube-api-access-q6582" (OuterVolumeSpecName: "kube-api-access-q6582") pod "68e945ce-02ea-4a56-b338-7c769d1e3553" (UID: "68e945ce-02ea-4a56-b338-7c769d1e3553"). InnerVolumeSpecName "kube-api-access-q6582". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 00:05:08.331425 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.331407 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "68e945ce-02ea-4a56-b338-7c769d1e3553" (UID: "68e945ce-02ea-4a56-b338-7c769d1e3553"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 00:05:08.430164 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.430128 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-token\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:05:08.430164 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.430159 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68e945ce-02ea-4a56-b338-7c769d1e3553-istiod-ca-cert\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:05:08.430164 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.430168 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-podinfo\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:05:08.430386 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.430176 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-workload-socket\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:05:08.430386 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.430185 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6582\" (UniqueName: \"kubernetes.io/projected/68e945ce-02ea-4a56-b338-7c769d1e3553-kube-api-access-q6582\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:05:08.430386 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:08.430194 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68e945ce-02ea-4a56-b338-7c769d1e3553-istio-envoy\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:05:09.030940 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:09.030902 2568 generic.go:356] "Generic (PLEG): container finished" podID="68e945ce-02ea-4a56-b338-7c769d1e3553" containerID="6d9e4db8a987b9cddbf7609dadaf1511d90e11c5c2dea0aa7e81a577950db5a8" exitCode=0 Sep 19 00:05:09.031094 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:09.030949 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" Sep 19 00:05:09.031094 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:09.030995 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" event={"ID":"68e945ce-02ea-4a56-b338-7c769d1e3553","Type":"ContainerDied","Data":"6d9e4db8a987b9cddbf7609dadaf1511d90e11c5c2dea0aa7e81a577950db5a8"} Sep 19 00:05:09.031094 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:09.031046 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb" event={"ID":"68e945ce-02ea-4a56-b338-7c769d1e3553","Type":"ContainerDied","Data":"c0715de20ab69e727f5f79bff2d8fb6bdb6065c6a5b5fd0cf711348d38cac607"} Sep 19 00:05:09.031094 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:09.031069 2568 scope.go:117] "RemoveContainer" containerID="6d9e4db8a987b9cddbf7609dadaf1511d90e11c5c2dea0aa7e81a577950db5a8" Sep 19 00:05:09.039864 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:09.039838 2568 scope.go:117] "RemoveContainer" containerID="6d9e4db8a987b9cddbf7609dadaf1511d90e11c5c2dea0aa7e81a577950db5a8" Sep 19 00:05:09.040147 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:05:09.040130 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d9e4db8a987b9cddbf7609dadaf1511d90e11c5c2dea0aa7e81a577950db5a8\": container with ID starting with 6d9e4db8a987b9cddbf7609dadaf1511d90e11c5c2dea0aa7e81a577950db5a8 not found: ID does not exist" containerID="6d9e4db8a987b9cddbf7609dadaf1511d90e11c5c2dea0aa7e81a577950db5a8" Sep 19 00:05:09.040190 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:09.040158 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d9e4db8a987b9cddbf7609dadaf1511d90e11c5c2dea0aa7e81a577950db5a8"} err="failed to get container status \"6d9e4db8a987b9cddbf7609dadaf1511d90e11c5c2dea0aa7e81a577950db5a8\": rpc error: code = NotFound desc = could not find container \"6d9e4db8a987b9cddbf7609dadaf1511d90e11c5c2dea0aa7e81a577950db5a8\": container with ID starting with 6d9e4db8a987b9cddbf7609dadaf1511d90e11c5c2dea0aa7e81a577950db5a8 not found: ID does not exist" Sep 19 00:05:09.051343 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:09.051323 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb"] Sep 19 00:05:09.054673 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:09.054652 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-87hrb"] Sep 19 00:05:09.379340 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:09.379311 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68e945ce-02ea-4a56-b338-7c769d1e3553" path="/var/lib/kubelet/pods/68e945ce-02ea-4a56-b338-7c769d1e3553/volumes" Sep 19 00:05:11.993940 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:11.993904 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-kwnhz" Sep 19 00:05:13.001532 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:13.001509 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-ksrzk" Sep 19 00:05:16.015267 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:16.015235 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7968c85699-z2mnc" Sep 19 00:05:19.201892 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:19.201855 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d48994b45-f65t4"] Sep 19 00:05:44.221187 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.221139 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d48994b45-f65t4" podUID="e511b452-1df5-45a0-8a63-3361389068cb" containerName="console" containerID="cri-o://bd8fdf4170c402ac29f097f8b4648d15f087e7d251bb65efedc3f73db715eb50" gracePeriod=15 Sep 19 00:05:44.468977 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.468952 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:05:44.523959 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.523873 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-console-config\") pod \"e511b452-1df5-45a0-8a63-3361389068cb\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " Sep 19 00:05:44.523959 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.523925 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-trusted-ca-bundle\") pod \"e511b452-1df5-45a0-8a63-3361389068cb\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " Sep 19 00:05:44.523959 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.523961 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qmfz\" (UniqueName: \"kubernetes.io/projected/e511b452-1df5-45a0-8a63-3361389068cb-kube-api-access-7qmfz\") pod \"e511b452-1df5-45a0-8a63-3361389068cb\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " Sep 19 00:05:44.524228 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.524058 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e511b452-1df5-45a0-8a63-3361389068cb-console-serving-cert\") pod \"e511b452-1df5-45a0-8a63-3361389068cb\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " Sep 19 00:05:44.524228 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.524103 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e511b452-1df5-45a0-8a63-3361389068cb-console-oauth-config\") pod \"e511b452-1df5-45a0-8a63-3361389068cb\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " Sep 19 00:05:44.524228 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.524143 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-oauth-serving-cert\") pod \"e511b452-1df5-45a0-8a63-3361389068cb\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " Sep 19 00:05:44.524228 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.524164 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-service-ca\") pod \"e511b452-1df5-45a0-8a63-3361389068cb\" (UID: \"e511b452-1df5-45a0-8a63-3361389068cb\") " Sep 19 00:05:44.524463 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.524312 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-console-config" (OuterVolumeSpecName: "console-config") pod "e511b452-1df5-45a0-8a63-3361389068cb" (UID: "e511b452-1df5-45a0-8a63-3361389068cb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 00:05:44.524463 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.524373 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e511b452-1df5-45a0-8a63-3361389068cb" (UID: "e511b452-1df5-45a0-8a63-3361389068cb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 00:05:44.524567 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.524469 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-trusted-ca-bundle\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:05:44.524567 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.524487 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-console-config\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:05:44.524728 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.524700 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e511b452-1df5-45a0-8a63-3361389068cb" (UID: "e511b452-1df5-45a0-8a63-3361389068cb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 00:05:44.524819 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.524744 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-service-ca" (OuterVolumeSpecName: "service-ca") pod "e511b452-1df5-45a0-8a63-3361389068cb" (UID: "e511b452-1df5-45a0-8a63-3361389068cb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 00:05:44.526336 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.526311 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e511b452-1df5-45a0-8a63-3361389068cb-kube-api-access-7qmfz" (OuterVolumeSpecName: "kube-api-access-7qmfz") pod "e511b452-1df5-45a0-8a63-3361389068cb" (UID: "e511b452-1df5-45a0-8a63-3361389068cb"). InnerVolumeSpecName "kube-api-access-7qmfz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 00:05:44.526336 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.526329 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e511b452-1df5-45a0-8a63-3361389068cb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e511b452-1df5-45a0-8a63-3361389068cb" (UID: "e511b452-1df5-45a0-8a63-3361389068cb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 00:05:44.526464 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.526347 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e511b452-1df5-45a0-8a63-3361389068cb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e511b452-1df5-45a0-8a63-3361389068cb" (UID: "e511b452-1df5-45a0-8a63-3361389068cb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 00:05:44.625033 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.625001 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qmfz\" (UniqueName: \"kubernetes.io/projected/e511b452-1df5-45a0-8a63-3361389068cb-kube-api-access-7qmfz\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:05:44.625033 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.625031 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e511b452-1df5-45a0-8a63-3361389068cb-console-serving-cert\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:05:44.625033 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.625041 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e511b452-1df5-45a0-8a63-3361389068cb-console-oauth-config\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:05:44.625247 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.625050 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-oauth-serving-cert\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:05:44.625247 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:44.625059 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e511b452-1df5-45a0-8a63-3361389068cb-service-ca\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:05:45.188906 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:45.188870 2568 generic.go:356] "Generic (PLEG): container finished" podID="e511b452-1df5-45a0-8a63-3361389068cb" containerID="bd8fdf4170c402ac29f097f8b4648d15f087e7d251bb65efedc3f73db715eb50" exitCode=0 Sep 19 00:05:45.189083 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:45.188913 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d48994b45-f65t4" Sep 19 00:05:45.189083 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:45.188941 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d48994b45-f65t4" event={"ID":"e511b452-1df5-45a0-8a63-3361389068cb","Type":"ContainerDied","Data":"bd8fdf4170c402ac29f097f8b4648d15f087e7d251bb65efedc3f73db715eb50"} Sep 19 00:05:45.189083 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:45.188982 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d48994b45-f65t4" event={"ID":"e511b452-1df5-45a0-8a63-3361389068cb","Type":"ContainerDied","Data":"bba04d506af656705dea868876f648bfd532d8097e32d7380757a0c97bf6bb28"} Sep 19 00:05:45.189083 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:45.189003 2568 scope.go:117] "RemoveContainer" containerID="bd8fdf4170c402ac29f097f8b4648d15f087e7d251bb65efedc3f73db715eb50" Sep 19 00:05:45.198203 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:45.198184 2568 scope.go:117] "RemoveContainer" containerID="bd8fdf4170c402ac29f097f8b4648d15f087e7d251bb65efedc3f73db715eb50" Sep 19 00:05:45.198471 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:05:45.198454 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd8fdf4170c402ac29f097f8b4648d15f087e7d251bb65efedc3f73db715eb50\": container with ID starting with bd8fdf4170c402ac29f097f8b4648d15f087e7d251bb65efedc3f73db715eb50 not found: ID does not exist" containerID="bd8fdf4170c402ac29f097f8b4648d15f087e7d251bb65efedc3f73db715eb50" Sep 19 00:05:45.198521 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:45.198479 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd8fdf4170c402ac29f097f8b4648d15f087e7d251bb65efedc3f73db715eb50"} err="failed to get container status \"bd8fdf4170c402ac29f097f8b4648d15f087e7d251bb65efedc3f73db715eb50\": rpc error: code = NotFound desc = could not find container \"bd8fdf4170c402ac29f097f8b4648d15f087e7d251bb65efedc3f73db715eb50\": container with ID starting with bd8fdf4170c402ac29f097f8b4648d15f087e7d251bb65efedc3f73db715eb50 not found: ID does not exist" Sep 19 00:05:45.209507 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:45.209477 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d48994b45-f65t4"] Sep 19 00:05:45.213354 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:45.213333 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d48994b45-f65t4"] Sep 19 00:05:45.379583 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:05:45.379543 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e511b452-1df5-45a0-8a63-3361389068cb" path="/var/lib/kubelet/pods/e511b452-1df5-45a0-8a63-3361389068cb/volumes" Sep 19 00:06:26.153887 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.153854 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8"] Sep 19 00:06:26.154325 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.154167 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68e945ce-02ea-4a56-b338-7c769d1e3553" containerName="istio-proxy" Sep 19 00:06:26.154325 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.154178 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="68e945ce-02ea-4a56-b338-7c769d1e3553" containerName="istio-proxy" Sep 19 00:06:26.154325 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.154195 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e511b452-1df5-45a0-8a63-3361389068cb" containerName="console" Sep 19 00:06:26.154325 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.154201 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e511b452-1df5-45a0-8a63-3361389068cb" containerName="console" Sep 19 00:06:26.154325 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.154250 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="68e945ce-02ea-4a56-b338-7c769d1e3553" containerName="istio-proxy" Sep 19 00:06:26.154325 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.154258 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e511b452-1df5-45a0-8a63-3361389068cb" containerName="console" Sep 19 00:06:26.155902 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.155883 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8" Sep 19 00:06:26.158402 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.158374 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 19 00:06:26.158543 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.158418 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Sep 19 00:06:26.158543 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.158422 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 19 00:06:26.159357 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.159340 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Sep 19 00:06:26.159441 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.159365 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rfshb\"" Sep 19 00:06:26.164764 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.164742 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8"] Sep 19 00:06:26.249974 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.249936 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c407d56b-6c17-433f-b37b-942ae38f1a0c-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-cdbw8\" (UID: \"c407d56b-6c17-433f-b37b-942ae38f1a0c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8" Sep 19 00:06:26.250152 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.249984 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b99bc\" (UniqueName: \"kubernetes.io/projected/c407d56b-6c17-433f-b37b-942ae38f1a0c-kube-api-access-b99bc\") pod \"kuadrant-console-plugin-fbdd4f445-cdbw8\" (UID: \"c407d56b-6c17-433f-b37b-942ae38f1a0c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8" Sep 19 00:06:26.250152 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.250045 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c407d56b-6c17-433f-b37b-942ae38f1a0c-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-cdbw8\" (UID: \"c407d56b-6c17-433f-b37b-942ae38f1a0c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8" Sep 19 00:06:26.351066 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.351033 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b99bc\" (UniqueName: \"kubernetes.io/projected/c407d56b-6c17-433f-b37b-942ae38f1a0c-kube-api-access-b99bc\") pod \"kuadrant-console-plugin-fbdd4f445-cdbw8\" (UID: \"c407d56b-6c17-433f-b37b-942ae38f1a0c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8" Sep 19 00:06:26.351254 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.351082 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c407d56b-6c17-433f-b37b-942ae38f1a0c-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-cdbw8\" (UID: \"c407d56b-6c17-433f-b37b-942ae38f1a0c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8" Sep 19 00:06:26.351254 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.351131 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c407d56b-6c17-433f-b37b-942ae38f1a0c-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-cdbw8\" (UID: \"c407d56b-6c17-433f-b37b-942ae38f1a0c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8" Sep 19 00:06:26.351713 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.351693 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c407d56b-6c17-433f-b37b-942ae38f1a0c-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-cdbw8\" (UID: \"c407d56b-6c17-433f-b37b-942ae38f1a0c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8" Sep 19 00:06:26.353709 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.353684 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c407d56b-6c17-433f-b37b-942ae38f1a0c-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-cdbw8\" (UID: \"c407d56b-6c17-433f-b37b-942ae38f1a0c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8" Sep 19 00:06:26.358692 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.358667 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b99bc\" (UniqueName: \"kubernetes.io/projected/c407d56b-6c17-433f-b37b-942ae38f1a0c-kube-api-access-b99bc\") pod \"kuadrant-console-plugin-fbdd4f445-cdbw8\" (UID: \"c407d56b-6c17-433f-b37b-942ae38f1a0c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8" Sep 19 00:06:26.465928 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.465853 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8" Sep 19 00:06:26.733284 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:26.733260 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8"] Sep 19 00:06:26.735601 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:06:26.735576 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc407d56b_6c17_433f_b37b_942ae38f1a0c.slice/crio-4a218d00cafca3cb7fcc747716a3831fbfc66e62189adeb55915cee9722b3073 WatchSource:0}: Error finding container 4a218d00cafca3cb7fcc747716a3831fbfc66e62189adeb55915cee9722b3073: Status 404 returned error can't find the container with id 4a218d00cafca3cb7fcc747716a3831fbfc66e62189adeb55915cee9722b3073 Sep 19 00:06:27.372980 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:27.372948 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8" event={"ID":"c407d56b-6c17-433f-b37b-942ae38f1a0c","Type":"ContainerStarted","Data":"4a218d00cafca3cb7fcc747716a3831fbfc66e62189adeb55915cee9722b3073"} Sep 19 00:06:31.391837 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:31.391782 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8" event={"ID":"c407d56b-6c17-433f-b37b-942ae38f1a0c","Type":"ContainerStarted","Data":"48486f98d6b009c2d33edff939b4f66f38ca303f3cc7fb3c1b4b8e0d8ec85f54"} Sep 19 00:06:31.407960 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:06:31.407860 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdbw8" podStartSLOduration=0.934407537 podStartE2EDuration="5.407841089s" totalImagesPullingTime="4.473433552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:06:31.407199031 +0000 UTC m=+482.560792894" watchObservedRunningTime="2026-09-19 00:06:31.407841089 +0000 UTC m=+482.561434955" Sep 19 00:07:25.013277 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.013198 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-bdn2k"] Sep 19 00:07:25.015308 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.015291 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-bdn2k" Sep 19 00:07:25.017636 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.017618 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bfd78\"" Sep 19 00:07:25.022626 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.022595 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-bdn2k"] Sep 19 00:07:25.125135 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.125097 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gftkf\" (UniqueName: \"kubernetes.io/projected/1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef-kube-api-access-gftkf\") pod \"authorino-7bcfd8f9cd-bdn2k\" (UID: \"1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef\") " pod="kuadrant-system/authorino-7bcfd8f9cd-bdn2k" Sep 19 00:07:25.148275 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.148245 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-wtmkw"] Sep 19 00:07:25.150478 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.150457 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-wtmkw" Sep 19 00:07:25.157435 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.157408 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-wtmkw"] Sep 19 00:07:25.226331 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.226295 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96jn8\" (UniqueName: \"kubernetes.io/projected/a6c8ffab-c4eb-469a-8f1e-395c9e646ca8-kube-api-access-96jn8\") pod \"authorino-59c974bd48-wtmkw\" (UID: \"a6c8ffab-c4eb-469a-8f1e-395c9e646ca8\") " pod="kuadrant-system/authorino-59c974bd48-wtmkw" Sep 19 00:07:25.226492 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.226354 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gftkf\" (UniqueName: \"kubernetes.io/projected/1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef-kube-api-access-gftkf\") pod \"authorino-7bcfd8f9cd-bdn2k\" (UID: \"1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef\") " pod="kuadrant-system/authorino-7bcfd8f9cd-bdn2k" Sep 19 00:07:25.233784 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.233761 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gftkf\" (UniqueName: \"kubernetes.io/projected/1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef-kube-api-access-gftkf\") pod \"authorino-7bcfd8f9cd-bdn2k\" (UID: \"1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef\") " pod="kuadrant-system/authorino-7bcfd8f9cd-bdn2k" Sep 19 00:07:25.324716 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.324632 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-bdn2k" Sep 19 00:07:25.327710 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.327688 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96jn8\" (UniqueName: \"kubernetes.io/projected/a6c8ffab-c4eb-469a-8f1e-395c9e646ca8-kube-api-access-96jn8\") pod \"authorino-59c974bd48-wtmkw\" (UID: \"a6c8ffab-c4eb-469a-8f1e-395c9e646ca8\") " pod="kuadrant-system/authorino-59c974bd48-wtmkw" Sep 19 00:07:25.335810 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.335762 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96jn8\" (UniqueName: \"kubernetes.io/projected/a6c8ffab-c4eb-469a-8f1e-395c9e646ca8-kube-api-access-96jn8\") pod \"authorino-59c974bd48-wtmkw\" (UID: \"a6c8ffab-c4eb-469a-8f1e-395c9e646ca8\") " pod="kuadrant-system/authorino-59c974bd48-wtmkw" Sep 19 00:07:25.459575 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.459541 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-wtmkw" Sep 19 00:07:25.590109 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.590028 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-bdn2k"] Sep 19 00:07:25.593120 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:07:25.593093 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c37a5c7_1f39_4bc9_abfb_7a76d666a7ef.slice/crio-5cc71b1e43a8cdba3dc23166207bc7191605b7f6d76b8b92ef83b84a4e1633f9 WatchSource:0}: Error finding container 5cc71b1e43a8cdba3dc23166207bc7191605b7f6d76b8b92ef83b84a4e1633f9: Status 404 returned error can't find the container with id 5cc71b1e43a8cdba3dc23166207bc7191605b7f6d76b8b92ef83b84a4e1633f9 Sep 19 00:07:25.619497 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.619470 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-bdn2k" event={"ID":"1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef","Type":"ContainerStarted","Data":"5cc71b1e43a8cdba3dc23166207bc7191605b7f6d76b8b92ef83b84a4e1633f9"} Sep 19 00:07:25.724085 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:25.724056 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-wtmkw"] Sep 19 00:07:25.726616 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:07:25.726585 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6c8ffab_c4eb_469a_8f1e_395c9e646ca8.slice/crio-8adef8ea28617d848c0e4e0dde49a1a883f76dcfb92c28f85de9c3265c967066 WatchSource:0}: Error finding container 8adef8ea28617d848c0e4e0dde49a1a883f76dcfb92c28f85de9c3265c967066: Status 404 returned error can't find the container with id 8adef8ea28617d848c0e4e0dde49a1a883f76dcfb92c28f85de9c3265c967066 Sep 19 00:07:26.627220 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:26.627154 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-wtmkw" event={"ID":"a6c8ffab-c4eb-469a-8f1e-395c9e646ca8","Type":"ContainerStarted","Data":"8adef8ea28617d848c0e4e0dde49a1a883f76dcfb92c28f85de9c3265c967066"} Sep 19 00:07:28.638048 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:28.637938 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-wtmkw" event={"ID":"a6c8ffab-c4eb-469a-8f1e-395c9e646ca8","Type":"ContainerStarted","Data":"7df226460f0059f852f9dd1843d653104c2feed6fb45329e1fbcbbc691c92a33"} Sep 19 00:07:28.639448 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:28.639424 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-bdn2k" event={"ID":"1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef","Type":"ContainerStarted","Data":"60cc9a32de5719aee3abb49b40ac2639b90ca30ba858719afa54daf944514d95"} Sep 19 00:07:28.652821 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:28.652759 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-wtmkw" podStartSLOduration=0.998400161 podStartE2EDuration="3.652743106s" totalImagesPullingTime="2.654342945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:07:28.652338321 +0000 UTC m=+539.805932187" watchObservedRunningTime="2026-09-19 00:07:28.652743106 +0000 UTC m=+539.806336971" Sep 19 00:07:28.669585 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:28.669522 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-bdn2k" podStartSLOduration=1.89026119 podStartE2EDuration="4.669498109s" totalImagesPullingTime="2.779236919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:07:28.667917531 +0000 UTC m=+539.821511397" watchObservedRunningTime="2026-09-19 00:07:28.669498109 +0000 UTC m=+539.823091974" Sep 19 00:07:28.687355 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:28.687326 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-bdn2k"] Sep 19 00:07:30.646910 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:30.646870 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-bdn2k" podUID="1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef" containerName="authorino" containerID="cri-o://60cc9a32de5719aee3abb49b40ac2639b90ca30ba858719afa54daf944514d95" gracePeriod=30 Sep 19 00:07:30.887452 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:30.887430 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-bdn2k" Sep 19 00:07:30.974774 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:30.974697 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gftkf\" (UniqueName: \"kubernetes.io/projected/1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef-kube-api-access-gftkf\") pod \"1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef\" (UID: \"1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef\") " Sep 19 00:07:30.976974 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:30.976952 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef-kube-api-access-gftkf" (OuterVolumeSpecName: "kube-api-access-gftkf") pod "1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef" (UID: "1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef"). InnerVolumeSpecName "kube-api-access-gftkf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 00:07:31.076173 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:31.076135 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gftkf\" (UniqueName: \"kubernetes.io/projected/1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef-kube-api-access-gftkf\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:07:31.656605 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:31.652488 2568 generic.go:356] "Generic (PLEG): container finished" podID="1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef" containerID="60cc9a32de5719aee3abb49b40ac2639b90ca30ba858719afa54daf944514d95" exitCode=0 Sep 19 00:07:31.656605 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:31.652581 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-bdn2k" event={"ID":"1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef","Type":"ContainerDied","Data":"60cc9a32de5719aee3abb49b40ac2639b90ca30ba858719afa54daf944514d95"} Sep 19 00:07:31.656605 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:31.652634 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-bdn2k" event={"ID":"1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef","Type":"ContainerDied","Data":"5cc71b1e43a8cdba3dc23166207bc7191605b7f6d76b8b92ef83b84a4e1633f9"} Sep 19 00:07:31.656605 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:31.652659 2568 scope.go:117] "RemoveContainer" containerID="60cc9a32de5719aee3abb49b40ac2639b90ca30ba858719afa54daf944514d95" Sep 19 00:07:31.656605 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:31.652883 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-bdn2k" Sep 19 00:07:31.665546 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:31.665526 2568 scope.go:117] "RemoveContainer" containerID="60cc9a32de5719aee3abb49b40ac2639b90ca30ba858719afa54daf944514d95" Sep 19 00:07:31.665866 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:07:31.665839 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60cc9a32de5719aee3abb49b40ac2639b90ca30ba858719afa54daf944514d95\": container with ID starting with 60cc9a32de5719aee3abb49b40ac2639b90ca30ba858719afa54daf944514d95 not found: ID does not exist" containerID="60cc9a32de5719aee3abb49b40ac2639b90ca30ba858719afa54daf944514d95" Sep 19 00:07:31.665975 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:31.665869 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60cc9a32de5719aee3abb49b40ac2639b90ca30ba858719afa54daf944514d95"} err="failed to get container status \"60cc9a32de5719aee3abb49b40ac2639b90ca30ba858719afa54daf944514d95\": rpc error: code = NotFound desc = could not find container \"60cc9a32de5719aee3abb49b40ac2639b90ca30ba858719afa54daf944514d95\": container with ID starting with 60cc9a32de5719aee3abb49b40ac2639b90ca30ba858719afa54daf944514d95 not found: ID does not exist" Sep 19 00:07:31.668827 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:31.668782 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-bdn2k"] Sep 19 00:07:31.672286 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:31.672265 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-bdn2k"] Sep 19 00:07:33.379655 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:33.379624 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef" path="/var/lib/kubelet/pods/1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef/volumes" Sep 19 00:07:55.621866 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.621790 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-sjvmx"] Sep 19 00:07:55.622266 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.622141 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef" containerName="authorino" Sep 19 00:07:55.622266 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.622153 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef" containerName="authorino" Sep 19 00:07:55.622266 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.622222 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c37a5c7-1f39-4bc9-abfb-7a76d666a7ef" containerName="authorino" Sep 19 00:07:55.624042 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.624025 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-sjvmx" Sep 19 00:07:55.631711 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.631685 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-sjvmx"] Sep 19 00:07:55.681250 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.681211 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z82ll\" (UniqueName: \"kubernetes.io/projected/d23e5945-712b-40e0-9575-835dcf5ec5ab-kube-api-access-z82ll\") pod \"authorino-557f876f69-sjvmx\" (UID: \"d23e5945-712b-40e0-9575-835dcf5ec5ab\") " pod="kuadrant-system/authorino-557f876f69-sjvmx" Sep 19 00:07:55.782471 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.782434 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z82ll\" (UniqueName: \"kubernetes.io/projected/d23e5945-712b-40e0-9575-835dcf5ec5ab-kube-api-access-z82ll\") pod \"authorino-557f876f69-sjvmx\" (UID: \"d23e5945-712b-40e0-9575-835dcf5ec5ab\") " pod="kuadrant-system/authorino-557f876f69-sjvmx" Sep 19 00:07:55.791189 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.791158 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z82ll\" (UniqueName: \"kubernetes.io/projected/d23e5945-712b-40e0-9575-835dcf5ec5ab-kube-api-access-z82ll\") pod \"authorino-557f876f69-sjvmx\" (UID: \"d23e5945-712b-40e0-9575-835dcf5ec5ab\") " pod="kuadrant-system/authorino-557f876f69-sjvmx" Sep 19 00:07:55.852410 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.852370 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-sjvmx"] Sep 19 00:07:55.852668 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.852650 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-sjvmx" Sep 19 00:07:55.874487 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.874426 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6785986d88-cwtt8"] Sep 19 00:07:55.876757 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.876738 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6785986d88-cwtt8" Sep 19 00:07:55.883134 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.883101 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnclv\" (UniqueName: \"kubernetes.io/projected/97f6ab0c-5060-40f4-9adc-e9aa0b6658c9-kube-api-access-pnclv\") pod \"authorino-6785986d88-cwtt8\" (UID: \"97f6ab0c-5060-40f4-9adc-e9aa0b6658c9\") " pod="kuadrant-system/authorino-6785986d88-cwtt8" Sep 19 00:07:55.885326 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.885302 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6785986d88-cwtt8"] Sep 19 00:07:55.920988 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.920935 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6785986d88-cwtt8"] Sep 19 00:07:55.921270 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:07:55.921243 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-pnclv], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6785986d88-cwtt8" podUID="97f6ab0c-5060-40f4-9adc-e9aa0b6658c9" Sep 19 00:07:55.944779 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.944748 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-cb95ff958-q6pm2"] Sep 19 00:07:55.946930 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.946909 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb95ff958-q6pm2" Sep 19 00:07:55.949238 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.949216 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Sep 19 00:07:55.955186 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.955165 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cb95ff958-q6pm2"] Sep 19 00:07:55.984277 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.984248 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnclv\" (UniqueName: \"kubernetes.io/projected/97f6ab0c-5060-40f4-9adc-e9aa0b6658c9-kube-api-access-pnclv\") pod \"authorino-6785986d88-cwtt8\" (UID: \"97f6ab0c-5060-40f4-9adc-e9aa0b6658c9\") " pod="kuadrant-system/authorino-6785986d88-cwtt8" Sep 19 00:07:55.991695 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:55.991672 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnclv\" (UniqueName: \"kubernetes.io/projected/97f6ab0c-5060-40f4-9adc-e9aa0b6658c9-kube-api-access-pnclv\") pod \"authorino-6785986d88-cwtt8\" (UID: \"97f6ab0c-5060-40f4-9adc-e9aa0b6658c9\") " pod="kuadrant-system/authorino-6785986d88-cwtt8" Sep 19 00:07:56.085579 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.085543 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad79f65a-fc08-4cf4-82ff-88ad4880cd6c-tls-cert\") pod \"authorino-cb95ff958-q6pm2\" (UID: \"ad79f65a-fc08-4cf4-82ff-88ad4880cd6c\") " pod="kuadrant-system/authorino-cb95ff958-q6pm2" Sep 19 00:07:56.085579 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.085577 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmgmb\" (UniqueName: \"kubernetes.io/projected/ad79f65a-fc08-4cf4-82ff-88ad4880cd6c-kube-api-access-rmgmb\") pod \"authorino-cb95ff958-q6pm2\" (UID: \"ad79f65a-fc08-4cf4-82ff-88ad4880cd6c\") " pod="kuadrant-system/authorino-cb95ff958-q6pm2" Sep 19 00:07:56.124171 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.124144 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-sjvmx"] Sep 19 00:07:56.127084 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:07:56.127061 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd23e5945_712b_40e0_9575_835dcf5ec5ab.slice/crio-014c00b0ff15e2579e179e48d0292d65f91ad9ffe48a3cc74f0a7bb6d255c508 WatchSource:0}: Error finding container 014c00b0ff15e2579e179e48d0292d65f91ad9ffe48a3cc74f0a7bb6d255c508: Status 404 returned error can't find the container with id 014c00b0ff15e2579e179e48d0292d65f91ad9ffe48a3cc74f0a7bb6d255c508 Sep 19 00:07:56.185989 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.185965 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad79f65a-fc08-4cf4-82ff-88ad4880cd6c-tls-cert\") pod \"authorino-cb95ff958-q6pm2\" (UID: \"ad79f65a-fc08-4cf4-82ff-88ad4880cd6c\") " pod="kuadrant-system/authorino-cb95ff958-q6pm2" Sep 19 00:07:56.186111 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.186002 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmgmb\" (UniqueName: \"kubernetes.io/projected/ad79f65a-fc08-4cf4-82ff-88ad4880cd6c-kube-api-access-rmgmb\") pod \"authorino-cb95ff958-q6pm2\" (UID: \"ad79f65a-fc08-4cf4-82ff-88ad4880cd6c\") " pod="kuadrant-system/authorino-cb95ff958-q6pm2" Sep 19 00:07:56.188425 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.188399 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad79f65a-fc08-4cf4-82ff-88ad4880cd6c-tls-cert\") pod \"authorino-cb95ff958-q6pm2\" (UID: \"ad79f65a-fc08-4cf4-82ff-88ad4880cd6c\") " pod="kuadrant-system/authorino-cb95ff958-q6pm2" Sep 19 00:07:56.194589 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.194566 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmgmb\" (UniqueName: \"kubernetes.io/projected/ad79f65a-fc08-4cf4-82ff-88ad4880cd6c-kube-api-access-rmgmb\") pod \"authorino-cb95ff958-q6pm2\" (UID: \"ad79f65a-fc08-4cf4-82ff-88ad4880cd6c\") " pod="kuadrant-system/authorino-cb95ff958-q6pm2" Sep 19 00:07:56.256082 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.256051 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb95ff958-q6pm2" Sep 19 00:07:56.522859 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.522834 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cb95ff958-q6pm2"] Sep 19 00:07:56.524971 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:07:56.524937 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad79f65a_fc08_4cf4_82ff_88ad4880cd6c.slice/crio-d649224834ee39c4602e9f0674f6e4a98181ea102c47b21aec5950d343c7d873 WatchSource:0}: Error finding container d649224834ee39c4602e9f0674f6e4a98181ea102c47b21aec5950d343c7d873: Status 404 returned error can't find the container with id d649224834ee39c4602e9f0674f6e4a98181ea102c47b21aec5950d343c7d873 Sep 19 00:07:56.756221 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.756185 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cb95ff958-q6pm2" event={"ID":"ad79f65a-fc08-4cf4-82ff-88ad4880cd6c","Type":"ContainerStarted","Data":"d649224834ee39c4602e9f0674f6e4a98181ea102c47b21aec5950d343c7d873"} Sep 19 00:07:56.757668 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.757639 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-sjvmx" event={"ID":"d23e5945-712b-40e0-9575-835dcf5ec5ab","Type":"ContainerStarted","Data":"c7ab0c7d3eff4c12dc68989804da6f4b5d99e831263560f23b3179d5561f17ba"} Sep 19 00:07:56.757812 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.757680 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-sjvmx" event={"ID":"d23e5945-712b-40e0-9575-835dcf5ec5ab","Type":"ContainerStarted","Data":"014c00b0ff15e2579e179e48d0292d65f91ad9ffe48a3cc74f0a7bb6d255c508"} Sep 19 00:07:56.757812 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.757650 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6785986d88-cwtt8" Sep 19 00:07:56.757812 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.757702 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-sjvmx" podUID="d23e5945-712b-40e0-9575-835dcf5ec5ab" containerName="authorino" containerID="cri-o://c7ab0c7d3eff4c12dc68989804da6f4b5d99e831263560f23b3179d5561f17ba" gracePeriod=30 Sep 19 00:07:56.763811 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.763780 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6785986d88-cwtt8" Sep 19 00:07:56.771729 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.771654 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-sjvmx" podStartSLOduration=1.3807637 podStartE2EDuration="1.771639268s" totalImagesPullingTime="390.875568ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:07:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:07:56.770953251 +0000 UTC m=+567.924547115" watchObservedRunningTime="2026-09-19 00:07:56.771639268 +0000 UTC m=+567.925233133" Sep 19 00:07:56.789988 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.789958 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnclv\" (UniqueName: \"kubernetes.io/projected/97f6ab0c-5060-40f4-9adc-e9aa0b6658c9-kube-api-access-pnclv\") pod \"97f6ab0c-5060-40f4-9adc-e9aa0b6658c9\" (UID: \"97f6ab0c-5060-40f4-9adc-e9aa0b6658c9\") " Sep 19 00:07:56.792051 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.792030 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97f6ab0c-5060-40f4-9adc-e9aa0b6658c9-kube-api-access-pnclv" (OuterVolumeSpecName: "kube-api-access-pnclv") pod "97f6ab0c-5060-40f4-9adc-e9aa0b6658c9" (UID: "97f6ab0c-5060-40f4-9adc-e9aa0b6658c9"). InnerVolumeSpecName "kube-api-access-pnclv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 00:07:56.891194 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:56.891165 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnclv\" (UniqueName: \"kubernetes.io/projected/97f6ab0c-5060-40f4-9adc-e9aa0b6658c9-kube-api-access-pnclv\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:07:57.025651 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.025629 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-sjvmx" Sep 19 00:07:57.093331 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.093299 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z82ll\" (UniqueName: \"kubernetes.io/projected/d23e5945-712b-40e0-9575-835dcf5ec5ab-kube-api-access-z82ll\") pod \"d23e5945-712b-40e0-9575-835dcf5ec5ab\" (UID: \"d23e5945-712b-40e0-9575-835dcf5ec5ab\") " Sep 19 00:07:57.095787 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.095725 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d23e5945-712b-40e0-9575-835dcf5ec5ab-kube-api-access-z82ll" (OuterVolumeSpecName: "kube-api-access-z82ll") pod "d23e5945-712b-40e0-9575-835dcf5ec5ab" (UID: "d23e5945-712b-40e0-9575-835dcf5ec5ab"). InnerVolumeSpecName "kube-api-access-z82ll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 00:07:57.194180 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.194147 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z82ll\" (UniqueName: \"kubernetes.io/projected/d23e5945-712b-40e0-9575-835dcf5ec5ab-kube-api-access-z82ll\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:07:57.765000 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.764904 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cb95ff958-q6pm2" event={"ID":"ad79f65a-fc08-4cf4-82ff-88ad4880cd6c","Type":"ContainerStarted","Data":"9999efa1e5afb9551b8f8d5eebf8b28d5af6ee358649372eb001028380399176"} Sep 19 00:07:57.766117 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.766089 2568 generic.go:356] "Generic (PLEG): container finished" podID="d23e5945-712b-40e0-9575-835dcf5ec5ab" containerID="c7ab0c7d3eff4c12dc68989804da6f4b5d99e831263560f23b3179d5561f17ba" exitCode=0 Sep 19 00:07:57.766222 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.766130 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-sjvmx" Sep 19 00:07:57.766222 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.766161 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-sjvmx" event={"ID":"d23e5945-712b-40e0-9575-835dcf5ec5ab","Type":"ContainerDied","Data":"c7ab0c7d3eff4c12dc68989804da6f4b5d99e831263560f23b3179d5561f17ba"} Sep 19 00:07:57.766222 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.766193 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-sjvmx" event={"ID":"d23e5945-712b-40e0-9575-835dcf5ec5ab","Type":"ContainerDied","Data":"014c00b0ff15e2579e179e48d0292d65f91ad9ffe48a3cc74f0a7bb6d255c508"} Sep 19 00:07:57.766222 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.766212 2568 scope.go:117] "RemoveContainer" containerID="c7ab0c7d3eff4c12dc68989804da6f4b5d99e831263560f23b3179d5561f17ba" Sep 19 00:07:57.766384 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.766325 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6785986d88-cwtt8" Sep 19 00:07:57.774239 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.774222 2568 scope.go:117] "RemoveContainer" containerID="c7ab0c7d3eff4c12dc68989804da6f4b5d99e831263560f23b3179d5561f17ba" Sep 19 00:07:57.774502 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:07:57.774484 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7ab0c7d3eff4c12dc68989804da6f4b5d99e831263560f23b3179d5561f17ba\": container with ID starting with c7ab0c7d3eff4c12dc68989804da6f4b5d99e831263560f23b3179d5561f17ba not found: ID does not exist" containerID="c7ab0c7d3eff4c12dc68989804da6f4b5d99e831263560f23b3179d5561f17ba" Sep 19 00:07:57.774546 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.774510 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7ab0c7d3eff4c12dc68989804da6f4b5d99e831263560f23b3179d5561f17ba"} err="failed to get container status \"c7ab0c7d3eff4c12dc68989804da6f4b5d99e831263560f23b3179d5561f17ba\": rpc error: code = NotFound desc = could not find container \"c7ab0c7d3eff4c12dc68989804da6f4b5d99e831263560f23b3179d5561f17ba\": container with ID starting with c7ab0c7d3eff4c12dc68989804da6f4b5d99e831263560f23b3179d5561f17ba not found: ID does not exist" Sep 19 00:07:57.780848 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.780774 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-cb95ff958-q6pm2" podStartSLOduration=2.354659903 podStartE2EDuration="2.780762259s" totalImagesPullingTime="426.102356ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:07:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:07:57.779177973 +0000 UTC m=+568.932771839" watchObservedRunningTime="2026-09-19 00:07:57.780762259 +0000 UTC m=+568.934356124" Sep 19 00:07:57.792951 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.792923 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-sjvmx"] Sep 19 00:07:57.798894 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.798861 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-sjvmx"] Sep 19 00:07:57.802236 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.802212 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-wtmkw"] Sep 19 00:07:57.802388 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.802368 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-wtmkw" podUID="a6c8ffab-c4eb-469a-8f1e-395c9e646ca8" containerName="authorino" containerID="cri-o://7df226460f0059f852f9dd1843d653104c2feed6fb45329e1fbcbbc691c92a33" gracePeriod=30 Sep 19 00:07:57.821577 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.821546 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6785986d88-cwtt8"] Sep 19 00:07:57.826909 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:57.826882 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6785986d88-cwtt8"] Sep 19 00:07:58.045993 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:58.045973 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-wtmkw" Sep 19 00:07:58.101249 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:58.101216 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96jn8\" (UniqueName: \"kubernetes.io/projected/a6c8ffab-c4eb-469a-8f1e-395c9e646ca8-kube-api-access-96jn8\") pod \"a6c8ffab-c4eb-469a-8f1e-395c9e646ca8\" (UID: \"a6c8ffab-c4eb-469a-8f1e-395c9e646ca8\") " Sep 19 00:07:58.103426 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:58.103401 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6c8ffab-c4eb-469a-8f1e-395c9e646ca8-kube-api-access-96jn8" (OuterVolumeSpecName: "kube-api-access-96jn8") pod "a6c8ffab-c4eb-469a-8f1e-395c9e646ca8" (UID: "a6c8ffab-c4eb-469a-8f1e-395c9e646ca8"). InnerVolumeSpecName "kube-api-access-96jn8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 00:07:58.201699 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:58.201661 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96jn8\" (UniqueName: \"kubernetes.io/projected/a6c8ffab-c4eb-469a-8f1e-395c9e646ca8-kube-api-access-96jn8\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:07:58.774458 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:58.774427 2568 generic.go:356] "Generic (PLEG): container finished" podID="a6c8ffab-c4eb-469a-8f1e-395c9e646ca8" containerID="7df226460f0059f852f9dd1843d653104c2feed6fb45329e1fbcbbc691c92a33" exitCode=0 Sep 19 00:07:58.774924 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:58.774472 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-wtmkw" Sep 19 00:07:58.774924 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:58.774517 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-wtmkw" event={"ID":"a6c8ffab-c4eb-469a-8f1e-395c9e646ca8","Type":"ContainerDied","Data":"7df226460f0059f852f9dd1843d653104c2feed6fb45329e1fbcbbc691c92a33"} Sep 19 00:07:58.774924 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:58.774563 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-wtmkw" event={"ID":"a6c8ffab-c4eb-469a-8f1e-395c9e646ca8","Type":"ContainerDied","Data":"8adef8ea28617d848c0e4e0dde49a1a883f76dcfb92c28f85de9c3265c967066"} Sep 19 00:07:58.774924 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:58.774590 2568 scope.go:117] "RemoveContainer" containerID="7df226460f0059f852f9dd1843d653104c2feed6fb45329e1fbcbbc691c92a33" Sep 19 00:07:58.782774 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:58.782759 2568 scope.go:117] "RemoveContainer" containerID="7df226460f0059f852f9dd1843d653104c2feed6fb45329e1fbcbbc691c92a33" Sep 19 00:07:58.783068 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:07:58.783049 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7df226460f0059f852f9dd1843d653104c2feed6fb45329e1fbcbbc691c92a33\": container with ID starting with 7df226460f0059f852f9dd1843d653104c2feed6fb45329e1fbcbbc691c92a33 not found: ID does not exist" containerID="7df226460f0059f852f9dd1843d653104c2feed6fb45329e1fbcbbc691c92a33" Sep 19 00:07:58.783107 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:58.783077 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7df226460f0059f852f9dd1843d653104c2feed6fb45329e1fbcbbc691c92a33"} err="failed to get container status \"7df226460f0059f852f9dd1843d653104c2feed6fb45329e1fbcbbc691c92a33\": rpc error: code = NotFound desc = could not find container \"7df226460f0059f852f9dd1843d653104c2feed6fb45329e1fbcbbc691c92a33\": container with ID starting with 7df226460f0059f852f9dd1843d653104c2feed6fb45329e1fbcbbc691c92a33 not found: ID does not exist" Sep 19 00:07:58.796139 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:58.796116 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-wtmkw"] Sep 19 00:07:58.799809 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:58.799771 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-wtmkw"] Sep 19 00:07:59.379483 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:59.379449 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97f6ab0c-5060-40f4-9adc-e9aa0b6658c9" path="/var/lib/kubelet/pods/97f6ab0c-5060-40f4-9adc-e9aa0b6658c9/volumes" Sep 19 00:07:59.379681 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:59.379669 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6c8ffab-c4eb-469a-8f1e-395c9e646ca8" path="/var/lib/kubelet/pods/a6c8ffab-c4eb-469a-8f1e-395c9e646ca8/volumes" Sep 19 00:07:59.379969 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:07:59.379957 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d23e5945-712b-40e0-9575-835dcf5ec5ab" path="/var/lib/kubelet/pods/d23e5945-712b-40e0-9575-835dcf5ec5ab/volumes" Sep 19 00:09:13.895082 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:13.895049 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml"] Sep 19 00:09:13.895545 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:13.895526 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d23e5945-712b-40e0-9575-835dcf5ec5ab" containerName="authorino" Sep 19 00:09:13.895587 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:13.895550 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23e5945-712b-40e0-9575-835dcf5ec5ab" containerName="authorino" Sep 19 00:09:13.895587 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:13.895566 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6c8ffab-c4eb-469a-8f1e-395c9e646ca8" containerName="authorino" Sep 19 00:09:13.895587 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:13.895574 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6c8ffab-c4eb-469a-8f1e-395c9e646ca8" containerName="authorino" Sep 19 00:09:13.895709 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:13.895641 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6c8ffab-c4eb-469a-8f1e-395c9e646ca8" containerName="authorino" Sep 19 00:09:13.895709 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:13.895654 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d23e5945-712b-40e0-9575-835dcf5ec5ab" containerName="authorino" Sep 19 00:09:13.897493 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:13.897477 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:13.900052 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:13.899998 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 19 00:09:13.900052 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:13.900039 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 19 00:09:13.901064 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:13.901045 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-shsj8\"" Sep 19 00:09:13.901204 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:13.901132 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 19 00:09:13.907954 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:13.907932 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml"] Sep 19 00:09:14.014363 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.014332 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96832456-a0ff-4f48-a35f-71a4d64b40a0-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.014363 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.014371 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wttpd\" (UniqueName: \"kubernetes.io/projected/96832456-a0ff-4f48-a35f-71a4d64b40a0-kube-api-access-wttpd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.014584 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.014399 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96832456-a0ff-4f48-a35f-71a4d64b40a0-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.014584 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.014520 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96832456-a0ff-4f48-a35f-71a4d64b40a0-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.014584 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.014552 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96832456-a0ff-4f48-a35f-71a4d64b40a0-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.014678 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.014586 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96832456-a0ff-4f48-a35f-71a4d64b40a0-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.115540 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.115509 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96832456-a0ff-4f48-a35f-71a4d64b40a0-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.115540 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.115542 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96832456-a0ff-4f48-a35f-71a4d64b40a0-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.115772 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.115578 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96832456-a0ff-4f48-a35f-71a4d64b40a0-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.115772 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.115653 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96832456-a0ff-4f48-a35f-71a4d64b40a0-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.115772 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.115686 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wttpd\" (UniqueName: \"kubernetes.io/projected/96832456-a0ff-4f48-a35f-71a4d64b40a0-kube-api-access-wttpd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.115772 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.115713 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96832456-a0ff-4f48-a35f-71a4d64b40a0-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.116010 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.115994 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96832456-a0ff-4f48-a35f-71a4d64b40a0-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.116066 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.116046 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96832456-a0ff-4f48-a35f-71a4d64b40a0-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.116147 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.116127 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96832456-a0ff-4f48-a35f-71a4d64b40a0-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.118089 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.118061 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96832456-a0ff-4f48-a35f-71a4d64b40a0-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.118317 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.118300 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96832456-a0ff-4f48-a35f-71a4d64b40a0-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.126393 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.126375 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wttpd\" (UniqueName: \"kubernetes.io/projected/96832456-a0ff-4f48-a35f-71a4d64b40a0-kube-api-access-wttpd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6bcml\" (UID: \"96832456-a0ff-4f48-a35f-71a4d64b40a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.208656 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.208581 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:14.475945 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.475922 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml"] Sep 19 00:09:14.477862 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:09:14.477836 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96832456_a0ff_4f48_a35f_71a4d64b40a0.slice/crio-115b26644927c7812b5f5fc1c369d4e9c9dc96a0781b6e0d6c4af4611283a617 WatchSource:0}: Error finding container 115b26644927c7812b5f5fc1c369d4e9c9dc96a0781b6e0d6c4af4611283a617: Status 404 returned error can't find the container with id 115b26644927c7812b5f5fc1c369d4e9c9dc96a0781b6e0d6c4af4611283a617 Sep 19 00:09:14.479159 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:14.479143 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 00:09:15.118660 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:15.118628 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" event={"ID":"96832456-a0ff-4f48-a35f-71a4d64b40a0","Type":"ContainerStarted","Data":"115b26644927c7812b5f5fc1c369d4e9c9dc96a0781b6e0d6c4af4611283a617"} Sep 19 00:09:16.124586 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:16.124551 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" event={"ID":"96832456-a0ff-4f48-a35f-71a4d64b40a0","Type":"ContainerStarted","Data":"3e72b42658c977c3f562542b76edaad2747bcf1e98303063b2499e57e2248ca5"} Sep 19 00:09:16.143060 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:16.143010 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" podStartSLOduration=2.166940188 podStartE2EDuration="3.142994766s" totalImagesPullingTime="976.054578ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:09:16.14236022 +0000 UTC m=+647.295954085" watchObservedRunningTime="2026-09-19 00:09:16.142994766 +0000 UTC m=+647.296588628" Sep 19 00:09:22.606808 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.606770 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l"] Sep 19 00:09:22.608871 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.608854 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.611381 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.611365 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 19 00:09:22.618758 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.618733 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l"] Sep 19 00:09:22.693482 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.693445 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.693482 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.693476 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqfwx\" (UniqueName: \"kubernetes.io/projected/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-kube-api-access-fqfwx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.693724 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.693624 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.693724 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.693708 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.693862 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.693733 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.693862 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.693762 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.795139 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.795101 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.795139 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.795136 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.795445 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.795156 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.795445 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.795175 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.795445 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.795191 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqfwx\" (UniqueName: \"kubernetes.io/projected/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-kube-api-access-fqfwx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.795445 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.795262 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.795653 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.795545 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.795690 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.795646 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.795724 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.795697 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.797628 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.797604 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.797837 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.797818 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.803158 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.803140 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqfwx\" (UniqueName: \"kubernetes.io/projected/4f5357dc-edb1-4cc8-a059-2d4f896d1cca-kube-api-access-fqfwx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r6q2l\" (UID: \"4f5357dc-edb1-4cc8-a059-2d4f896d1cca\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:22.918901 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:22.918869 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:23.186015 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:23.185970 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l"] Sep 19 00:09:23.188328 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:09:23.188301 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f5357dc_edb1_4cc8_a059_2d4f896d1cca.slice/crio-cc7aa12448c4cf7668213b833a7e18d1f03d4b625c23e577869a62ba433562af WatchSource:0}: Error finding container cc7aa12448c4cf7668213b833a7e18d1f03d4b625c23e577869a62ba433562af: Status 404 returned error can't find the container with id cc7aa12448c4cf7668213b833a7e18d1f03d4b625c23e577869a62ba433562af Sep 19 00:09:24.160127 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:24.160092 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" event={"ID":"4f5357dc-edb1-4cc8-a059-2d4f896d1cca","Type":"ContainerStarted","Data":"f54587ed4a28ed6c620b75a866ab0b3926a4a32a30961690cf44ad71da07d369"} Sep 19 00:09:24.160588 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:24.160153 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" event={"ID":"4f5357dc-edb1-4cc8-a059-2d4f896d1cca","Type":"ContainerStarted","Data":"cc7aa12448c4cf7668213b833a7e18d1f03d4b625c23e577869a62ba433562af"} Sep 19 00:09:24.177077 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:24.177023 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" podStartSLOduration=2.029838791 podStartE2EDuration="2.177007833s" totalImagesPullingTime="147.169042ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:09:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:09:24.174950092 +0000 UTC m=+655.328543973" watchObservedRunningTime="2026-09-19 00:09:24.177007833 +0000 UTC m=+655.330601697" Sep 19 00:09:24.208928 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:24.208893 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:24.209114 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:24.208953 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:24.224337 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:24.224308 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:25.175885 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:25.175854 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6bcml" Sep 19 00:09:32.919943 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:32.919910 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:32.920352 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:32.919964 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:32.932321 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:32.932289 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:33.209710 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:33.209622 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r6q2l" Sep 19 00:09:43.399358 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.399321 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd"] Sep 19 00:09:43.402682 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.402662 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.405253 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.405231 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Sep 19 00:09:43.412426 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.412384 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd"] Sep 19 00:09:43.571520 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.571480 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbvtg\" (UniqueName: \"kubernetes.io/projected/86a6dbef-3196-4476-b224-5e5777ba8b8f-kube-api-access-pbvtg\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.571676 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.571547 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86a6dbef-3196-4476-b224-5e5777ba8b8f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.571676 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.571607 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86a6dbef-3196-4476-b224-5e5777ba8b8f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.571676 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.571637 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86a6dbef-3196-4476-b224-5e5777ba8b8f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.571676 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.571661 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86a6dbef-3196-4476-b224-5e5777ba8b8f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.571840 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.571695 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86a6dbef-3196-4476-b224-5e5777ba8b8f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.673182 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.673098 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86a6dbef-3196-4476-b224-5e5777ba8b8f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.673182 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.673177 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbvtg\" (UniqueName: \"kubernetes.io/projected/86a6dbef-3196-4476-b224-5e5777ba8b8f-kube-api-access-pbvtg\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.673376 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.673219 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86a6dbef-3196-4476-b224-5e5777ba8b8f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.673376 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.673249 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86a6dbef-3196-4476-b224-5e5777ba8b8f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.673452 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.673389 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86a6dbef-3196-4476-b224-5e5777ba8b8f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.673452 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.673430 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86a6dbef-3196-4476-b224-5e5777ba8b8f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.673605 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.673582 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86a6dbef-3196-4476-b224-5e5777ba8b8f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.673672 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.673622 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86a6dbef-3196-4476-b224-5e5777ba8b8f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.673726 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.673686 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86a6dbef-3196-4476-b224-5e5777ba8b8f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.675833 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.675787 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86a6dbef-3196-4476-b224-5e5777ba8b8f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.676109 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.676091 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86a6dbef-3196-4476-b224-5e5777ba8b8f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.680195 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.680175 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbvtg\" (UniqueName: \"kubernetes.io/projected/86a6dbef-3196-4476-b224-5e5777ba8b8f-kube-api-access-pbvtg\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd\" (UID: \"86a6dbef-3196-4476-b224-5e5777ba8b8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.717524 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.717499 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:43.984456 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:43.984431 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd"] Sep 19 00:09:43.986771 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:09:43.986732 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86a6dbef_3196_4476_b224_5e5777ba8b8f.slice/crio-614137835def1249a4f91e0292b55223d4d479614749b379dd703498f9d6ba63 WatchSource:0}: Error finding container 614137835def1249a4f91e0292b55223d4d479614749b379dd703498f9d6ba63: Status 404 returned error can't find the container with id 614137835def1249a4f91e0292b55223d4d479614749b379dd703498f9d6ba63 Sep 19 00:09:44.248250 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:44.248166 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" event={"ID":"86a6dbef-3196-4476-b224-5e5777ba8b8f","Type":"ContainerStarted","Data":"4a6c829442445225af7373196cbc8365822c45414769ae8b5449a4a38fe7bed5"} Sep 19 00:09:44.248250 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:44.248210 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" event={"ID":"86a6dbef-3196-4476-b224-5e5777ba8b8f","Type":"ContainerStarted","Data":"614137835def1249a4f91e0292b55223d4d479614749b379dd703498f9d6ba63"} Sep 19 00:09:44.266408 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:44.266356 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" podStartSLOduration=1.106364458 podStartE2EDuration="1.266341259s" totalImagesPullingTime="159.976801ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:09:44.265437302 +0000 UTC m=+675.419031169" watchObservedRunningTime="2026-09-19 00:09:44.266341259 +0000 UTC m=+675.419935123" Sep 19 00:09:53.718100 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:53.718056 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:53.718100 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:53.718103 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:53.730418 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:53.730395 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:09:54.305368 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:09:54.305340 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c6gmd" Sep 19 00:11:02.570651 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:02.570566 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66dcdc5c67-kphc2"] Sep 19 00:11:02.572894 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:02.572879 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66dcdc5c67-kphc2" Sep 19 00:11:02.580492 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:02.580472 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66dcdc5c67-kphc2"] Sep 19 00:11:02.669499 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:02.669458 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e3490cac-f5b2-4ac7-9cdf-9a2667c9af29-tls-cert\") pod \"authorino-66dcdc5c67-kphc2\" (UID: \"e3490cac-f5b2-4ac7-9cdf-9a2667c9af29\") " pod="kuadrant-system/authorino-66dcdc5c67-kphc2" Sep 19 00:11:02.669672 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:02.669546 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs87k\" (UniqueName: \"kubernetes.io/projected/e3490cac-f5b2-4ac7-9cdf-9a2667c9af29-kube-api-access-cs87k\") pod \"authorino-66dcdc5c67-kphc2\" (UID: \"e3490cac-f5b2-4ac7-9cdf-9a2667c9af29\") " pod="kuadrant-system/authorino-66dcdc5c67-kphc2" Sep 19 00:11:02.770474 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:02.770439 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e3490cac-f5b2-4ac7-9cdf-9a2667c9af29-tls-cert\") pod \"authorino-66dcdc5c67-kphc2\" (UID: \"e3490cac-f5b2-4ac7-9cdf-9a2667c9af29\") " pod="kuadrant-system/authorino-66dcdc5c67-kphc2" Sep 19 00:11:02.770638 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:02.770494 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs87k\" (UniqueName: \"kubernetes.io/projected/e3490cac-f5b2-4ac7-9cdf-9a2667c9af29-kube-api-access-cs87k\") pod \"authorino-66dcdc5c67-kphc2\" (UID: \"e3490cac-f5b2-4ac7-9cdf-9a2667c9af29\") " pod="kuadrant-system/authorino-66dcdc5c67-kphc2" Sep 19 00:11:02.773132 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:02.773108 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e3490cac-f5b2-4ac7-9cdf-9a2667c9af29-tls-cert\") pod \"authorino-66dcdc5c67-kphc2\" (UID: \"e3490cac-f5b2-4ac7-9cdf-9a2667c9af29\") " pod="kuadrant-system/authorino-66dcdc5c67-kphc2" Sep 19 00:11:02.778034 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:02.778006 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs87k\" (UniqueName: \"kubernetes.io/projected/e3490cac-f5b2-4ac7-9cdf-9a2667c9af29-kube-api-access-cs87k\") pod \"authorino-66dcdc5c67-kphc2\" (UID: \"e3490cac-f5b2-4ac7-9cdf-9a2667c9af29\") " pod="kuadrant-system/authorino-66dcdc5c67-kphc2" Sep 19 00:11:02.882237 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:02.882204 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66dcdc5c67-kphc2" Sep 19 00:11:03.150093 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:03.150069 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66dcdc5c67-kphc2"] Sep 19 00:11:03.152617 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:11:03.152592 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3490cac_f5b2_4ac7_9cdf_9a2667c9af29.slice/crio-479984e6301a289c7c47e63bce6e42c3ba337f2220d402b63948e08f3fa4520e WatchSource:0}: Error finding container 479984e6301a289c7c47e63bce6e42c3ba337f2220d402b63948e08f3fa4520e: Status 404 returned error can't find the container with id 479984e6301a289c7c47e63bce6e42c3ba337f2220d402b63948e08f3fa4520e Sep 19 00:11:03.609105 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:03.609075 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66dcdc5c67-kphc2" event={"ID":"e3490cac-f5b2-4ac7-9cdf-9a2667c9af29","Type":"ContainerStarted","Data":"479984e6301a289c7c47e63bce6e42c3ba337f2220d402b63948e08f3fa4520e"} Sep 19 00:11:04.617331 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:04.617285 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66dcdc5c67-kphc2" event={"ID":"e3490cac-f5b2-4ac7-9cdf-9a2667c9af29","Type":"ContainerStarted","Data":"4db04ee16d9c0d54fc7b00e032be8c8031b6ff124f3735627b0f568d53c2127e"} Sep 19 00:11:04.637784 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:04.637719 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-66dcdc5c67-kphc2" podStartSLOduration=1.651007457 podStartE2EDuration="2.637696863s" totalImagesPullingTime="986.689406ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:11:04.635130256 +0000 UTC m=+755.788724112" watchObservedRunningTime="2026-09-19 00:11:04.637696863 +0000 UTC m=+755.791290728" Sep 19 00:11:04.664770 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:04.664735 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cb95ff958-q6pm2"] Sep 19 00:11:04.665237 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:04.665202 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-cb95ff958-q6pm2" podUID="ad79f65a-fc08-4cf4-82ff-88ad4880cd6c" containerName="authorino" containerID="cri-o://9999efa1e5afb9551b8f8d5eebf8b28d5af6ee358649372eb001028380399176" gracePeriod=30 Sep 19 00:11:04.923592 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:04.923567 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb95ff958-q6pm2" Sep 19 00:11:04.991571 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:04.991537 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad79f65a-fc08-4cf4-82ff-88ad4880cd6c-tls-cert\") pod \"ad79f65a-fc08-4cf4-82ff-88ad4880cd6c\" (UID: \"ad79f65a-fc08-4cf4-82ff-88ad4880cd6c\") " Sep 19 00:11:04.991737 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:04.991620 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmgmb\" (UniqueName: \"kubernetes.io/projected/ad79f65a-fc08-4cf4-82ff-88ad4880cd6c-kube-api-access-rmgmb\") pod \"ad79f65a-fc08-4cf4-82ff-88ad4880cd6c\" (UID: \"ad79f65a-fc08-4cf4-82ff-88ad4880cd6c\") " Sep 19 00:11:04.993809 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:04.993764 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad79f65a-fc08-4cf4-82ff-88ad4880cd6c-kube-api-access-rmgmb" (OuterVolumeSpecName: "kube-api-access-rmgmb") pod "ad79f65a-fc08-4cf4-82ff-88ad4880cd6c" (UID: "ad79f65a-fc08-4cf4-82ff-88ad4880cd6c"). InnerVolumeSpecName "kube-api-access-rmgmb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 00:11:05.002164 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:05.002129 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad79f65a-fc08-4cf4-82ff-88ad4880cd6c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ad79f65a-fc08-4cf4-82ff-88ad4880cd6c" (UID: "ad79f65a-fc08-4cf4-82ff-88ad4880cd6c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 00:11:05.093155 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:05.093103 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmgmb\" (UniqueName: \"kubernetes.io/projected/ad79f65a-fc08-4cf4-82ff-88ad4880cd6c-kube-api-access-rmgmb\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:11:05.093155 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:05.093148 2568 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad79f65a-fc08-4cf4-82ff-88ad4880cd6c-tls-cert\") on node \"ip-10-0-129-146.ec2.internal\" DevicePath \"\"" Sep 19 00:11:05.622854 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:05.622786 2568 generic.go:356] "Generic (PLEG): container finished" podID="ad79f65a-fc08-4cf4-82ff-88ad4880cd6c" containerID="9999efa1e5afb9551b8f8d5eebf8b28d5af6ee358649372eb001028380399176" exitCode=0 Sep 19 00:11:05.622854 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:05.622847 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb95ff958-q6pm2" Sep 19 00:11:05.623367 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:05.622875 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cb95ff958-q6pm2" event={"ID":"ad79f65a-fc08-4cf4-82ff-88ad4880cd6c","Type":"ContainerDied","Data":"9999efa1e5afb9551b8f8d5eebf8b28d5af6ee358649372eb001028380399176"} Sep 19 00:11:05.623367 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:05.622922 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cb95ff958-q6pm2" event={"ID":"ad79f65a-fc08-4cf4-82ff-88ad4880cd6c","Type":"ContainerDied","Data":"d649224834ee39c4602e9f0674f6e4a98181ea102c47b21aec5950d343c7d873"} Sep 19 00:11:05.623367 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:05.622948 2568 scope.go:117] "RemoveContainer" containerID="9999efa1e5afb9551b8f8d5eebf8b28d5af6ee358649372eb001028380399176" Sep 19 00:11:05.631670 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:05.631653 2568 scope.go:117] "RemoveContainer" containerID="9999efa1e5afb9551b8f8d5eebf8b28d5af6ee358649372eb001028380399176" Sep 19 00:11:05.632000 ip-10-0-129-146 kubenswrapper[2568]: E0919 00:11:05.631978 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9999efa1e5afb9551b8f8d5eebf8b28d5af6ee358649372eb001028380399176\": container with ID starting with 9999efa1e5afb9551b8f8d5eebf8b28d5af6ee358649372eb001028380399176 not found: ID does not exist" containerID="9999efa1e5afb9551b8f8d5eebf8b28d5af6ee358649372eb001028380399176" Sep 19 00:11:05.632088 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:05.632012 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9999efa1e5afb9551b8f8d5eebf8b28d5af6ee358649372eb001028380399176"} err="failed to get container status \"9999efa1e5afb9551b8f8d5eebf8b28d5af6ee358649372eb001028380399176\": rpc error: code = NotFound desc = could not find container \"9999efa1e5afb9551b8f8d5eebf8b28d5af6ee358649372eb001028380399176\": container with ID starting with 9999efa1e5afb9551b8f8d5eebf8b28d5af6ee358649372eb001028380399176 not found: ID does not exist" Sep 19 00:11:05.638097 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:05.638073 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cb95ff958-q6pm2"] Sep 19 00:11:05.641393 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:05.641330 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-cb95ff958-q6pm2"] Sep 19 00:11:07.379536 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:11:07.379501 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad79f65a-fc08-4cf4-82ff-88ad4880cd6c" path="/var/lib/kubelet/pods/ad79f65a-fc08-4cf4-82ff-88ad4880cd6c/volumes" Sep 19 00:18:36.750236 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:36.750198 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9kf4c/must-gather-j2b8b"] Sep 19 00:18:36.750855 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:36.750694 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad79f65a-fc08-4cf4-82ff-88ad4880cd6c" containerName="authorino" Sep 19 00:18:36.750855 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:36.750713 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad79f65a-fc08-4cf4-82ff-88ad4880cd6c" containerName="authorino" Sep 19 00:18:36.750855 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:36.750838 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad79f65a-fc08-4cf4-82ff-88ad4880cd6c" containerName="authorino" Sep 19 00:18:36.754109 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:36.754088 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9kf4c/must-gather-j2b8b" Sep 19 00:18:36.757193 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:36.757173 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9kf4c\"/\"openshift-service-ca.crt\"" Sep 19 00:18:36.758282 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:36.758265 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9kf4c\"/\"kube-root-ca.crt\"" Sep 19 00:18:36.758374 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:36.758271 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9kf4c\"/\"default-dockercfg-hdlp4\"" Sep 19 00:18:36.771867 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:36.771847 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9kf4c/must-gather-j2b8b"] Sep 19 00:18:36.802754 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:36.802729 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84phk\" (UniqueName: \"kubernetes.io/projected/879421c3-7248-4e49-ac39-41947e2a0955-kube-api-access-84phk\") pod \"must-gather-j2b8b\" (UID: \"879421c3-7248-4e49-ac39-41947e2a0955\") " pod="openshift-must-gather-9kf4c/must-gather-j2b8b" Sep 19 00:18:36.802870 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:36.802762 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/879421c3-7248-4e49-ac39-41947e2a0955-must-gather-output\") pod \"must-gather-j2b8b\" (UID: \"879421c3-7248-4e49-ac39-41947e2a0955\") " pod="openshift-must-gather-9kf4c/must-gather-j2b8b" Sep 19 00:18:36.903732 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:36.903698 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84phk\" (UniqueName: \"kubernetes.io/projected/879421c3-7248-4e49-ac39-41947e2a0955-kube-api-access-84phk\") pod \"must-gather-j2b8b\" (UID: \"879421c3-7248-4e49-ac39-41947e2a0955\") " pod="openshift-must-gather-9kf4c/must-gather-j2b8b" Sep 19 00:18:36.903895 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:36.903739 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/879421c3-7248-4e49-ac39-41947e2a0955-must-gather-output\") pod \"must-gather-j2b8b\" (UID: \"879421c3-7248-4e49-ac39-41947e2a0955\") " pod="openshift-must-gather-9kf4c/must-gather-j2b8b" Sep 19 00:18:36.904082 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:36.904066 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/879421c3-7248-4e49-ac39-41947e2a0955-must-gather-output\") pod \"must-gather-j2b8b\" (UID: \"879421c3-7248-4e49-ac39-41947e2a0955\") " pod="openshift-must-gather-9kf4c/must-gather-j2b8b" Sep 19 00:18:36.911301 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:36.911273 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84phk\" (UniqueName: \"kubernetes.io/projected/879421c3-7248-4e49-ac39-41947e2a0955-kube-api-access-84phk\") pod \"must-gather-j2b8b\" (UID: \"879421c3-7248-4e49-ac39-41947e2a0955\") " pod="openshift-must-gather-9kf4c/must-gather-j2b8b" Sep 19 00:18:37.062848 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:37.062766 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9kf4c/must-gather-j2b8b" Sep 19 00:18:37.331465 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:37.331422 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9kf4c/must-gather-j2b8b"] Sep 19 00:18:37.333728 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:18:37.333699 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod879421c3_7248_4e49_ac39_41947e2a0955.slice/crio-7a5ca1eed48126f6a2b9005fa42aeda6ab1da8fef8f8e6736e40ce157b8e682e WatchSource:0}: Error finding container 7a5ca1eed48126f6a2b9005fa42aeda6ab1da8fef8f8e6736e40ce157b8e682e: Status 404 returned error can't find the container with id 7a5ca1eed48126f6a2b9005fa42aeda6ab1da8fef8f8e6736e40ce157b8e682e Sep 19 00:18:37.335604 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:37.335584 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 00:18:37.614978 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:37.614947 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9kf4c/must-gather-j2b8b" event={"ID":"879421c3-7248-4e49-ac39-41947e2a0955","Type":"ContainerStarted","Data":"7a5ca1eed48126f6a2b9005fa42aeda6ab1da8fef8f8e6736e40ce157b8e682e"} Sep 19 00:18:38.622513 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:38.622481 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9kf4c/must-gather-j2b8b" event={"ID":"879421c3-7248-4e49-ac39-41947e2a0955","Type":"ContainerStarted","Data":"29db58b410802d5b354804fe0cb5574199a48333086cccc4dd0d4a49b7711106"} Sep 19 00:18:38.622906 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:38.622522 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9kf4c/must-gather-j2b8b" event={"ID":"879421c3-7248-4e49-ac39-41947e2a0955","Type":"ContainerStarted","Data":"13984b63a6d9905cd904b716b00e8259fb29b8c4800da5f00cd75c7f756f985b"} Sep 19 00:18:38.637099 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:38.637035 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9kf4c/must-gather-j2b8b" podStartSLOduration=1.780183981 podStartE2EDuration="2.637012684s" totalImagesPullingTime="856.828703ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 00:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:18:38.635679271 +0000 UTC m=+1209.789273140" watchObservedRunningTime="2026-09-19 00:18:38.637012684 +0000 UTC m=+1209.790606551" Sep 19 00:18:39.652190 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:39.652166 2568 ???:1] "http: TLS handshake error from 10.0.130.53:57324: EOF" Sep 19 00:18:39.656711 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:39.656688 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7nzpw_c29d9e92-fea6-47b8-b299-ec7ae98f3010/global-pull-secret-syncer/0.log" Sep 19 00:18:39.825574 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:39.825534 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qsdhf_9fe2a3fa-f8a3-4890-8bfc-2914fe2d336f/konnectivity-agent/0.log" Sep 19 00:18:39.844523 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:39.844493 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-146.ec2.internal_bae7cc99672b5dbd116a90c47b4ad4c3/haproxy/0.log" Sep 19 00:18:43.851496 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:43.851460 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-66dcdc5c67-kphc2_e3490cac-f5b2-4ac7-9cdf-9a2667c9af29/authorino/0.log" Sep 19 00:18:44.000321 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:44.000282 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-cdbw8_c407d56b-6c17-433f-b37b-942ae38f1a0c/kuadrant-console-plugin/0.log" Sep 19 00:18:45.806030 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:45.805941 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf/alertmanager/0.log" Sep 19 00:18:45.828853 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:45.828822 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf/config-reloader/0.log" Sep 19 00:18:45.854076 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:45.854052 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf/kube-rbac-proxy-web/0.log" Sep 19 00:18:45.877760 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:45.877710 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf/kube-rbac-proxy/0.log" Sep 19 00:18:45.904124 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:45.904091 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf/kube-rbac-proxy-metric/0.log" Sep 19 00:18:45.927491 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:45.927455 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf/prom-label-proxy/0.log" Sep 19 00:18:45.948367 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:45.948334 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9b6d489a-1018-4cdc-a5e4-ca0038ff9bcf/init-config-reloader/0.log" Sep 19 00:18:46.276815 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:46.276769 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ssw9v_e0f865cd-5066-4d6e-b11d-aa9ef1dd8702/node-exporter/0.log" Sep 19 00:18:46.300269 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:46.300201 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ssw9v_e0f865cd-5066-4d6e-b11d-aa9ef1dd8702/kube-rbac-proxy/0.log" Sep 19 00:18:46.320665 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:46.320630 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ssw9v_e0f865cd-5066-4d6e-b11d-aa9ef1dd8702/init-textfile/0.log" Sep 19 00:18:46.659450 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:46.659421 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b79cc9d87-c9l2q_d0e96afe-381b-42f2-915d-53555a0f6f1f/telemeter-client/0.log" Sep 19 00:18:46.685997 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:46.685953 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b79cc9d87-c9l2q_d0e96afe-381b-42f2-915d-53555a0f6f1f/reload/0.log" Sep 19 00:18:46.706568 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:46.706545 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b79cc9d87-c9l2q_d0e96afe-381b-42f2-915d-53555a0f6f1f/kube-rbac-proxy/0.log" Sep 19 00:18:49.847240 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:49.847192 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6594f4c75-p8mhg_338aaefa-b72b-46dd-976c-e8cb141eeba8/download-server/0.log" Sep 19 00:18:50.187464 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.187427 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb"] Sep 19 00:18:50.192965 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.192935 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.200590 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.200564 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb"] Sep 19 00:18:50.246200 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.246164 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cd471ea-70bb-4c9c-b866-674577a98819-lib-modules\") pod \"perf-node-gather-daemonset-plwhb\" (UID: \"3cd471ea-70bb-4c9c-b866-674577a98819\") " pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.246200 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.246208 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cd471ea-70bb-4c9c-b866-674577a98819-sys\") pod \"perf-node-gather-daemonset-plwhb\" (UID: \"3cd471ea-70bb-4c9c-b866-674577a98819\") " pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.246446 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.246234 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3cd471ea-70bb-4c9c-b866-674577a98819-proc\") pod \"perf-node-gather-daemonset-plwhb\" (UID: \"3cd471ea-70bb-4c9c-b866-674577a98819\") " pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.246446 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.246311 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3cd471ea-70bb-4c9c-b866-674577a98819-podres\") pod \"perf-node-gather-daemonset-plwhb\" (UID: \"3cd471ea-70bb-4c9c-b866-674577a98819\") " pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.246446 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.246389 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ns65\" (UniqueName: \"kubernetes.io/projected/3cd471ea-70bb-4c9c-b866-674577a98819-kube-api-access-7ns65\") pod \"perf-node-gather-daemonset-plwhb\" (UID: \"3cd471ea-70bb-4c9c-b866-674577a98819\") " pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.347607 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.347573 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cd471ea-70bb-4c9c-b866-674577a98819-lib-modules\") pod \"perf-node-gather-daemonset-plwhb\" (UID: \"3cd471ea-70bb-4c9c-b866-674577a98819\") " pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.347870 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.347851 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cd471ea-70bb-4c9c-b866-674577a98819-sys\") pod \"perf-node-gather-daemonset-plwhb\" (UID: \"3cd471ea-70bb-4c9c-b866-674577a98819\") " pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.348006 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.347992 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3cd471ea-70bb-4c9c-b866-674577a98819-proc\") pod \"perf-node-gather-daemonset-plwhb\" (UID: \"3cd471ea-70bb-4c9c-b866-674577a98819\") " pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.348209 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.348197 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3cd471ea-70bb-4c9c-b866-674577a98819-podres\") pod \"perf-node-gather-daemonset-plwhb\" (UID: \"3cd471ea-70bb-4c9c-b866-674577a98819\") " pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.348506 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.348492 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ns65\" (UniqueName: \"kubernetes.io/projected/3cd471ea-70bb-4c9c-b866-674577a98819-kube-api-access-7ns65\") pod \"perf-node-gather-daemonset-plwhb\" (UID: \"3cd471ea-70bb-4c9c-b866-674577a98819\") " pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.348990 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.348133 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3cd471ea-70bb-4c9c-b866-674577a98819-proc\") pod \"perf-node-gather-daemonset-plwhb\" (UID: \"3cd471ea-70bb-4c9c-b866-674577a98819\") " pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.349122 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.347925 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cd471ea-70bb-4c9c-b866-674577a98819-sys\") pod \"perf-node-gather-daemonset-plwhb\" (UID: \"3cd471ea-70bb-4c9c-b866-674577a98819\") " pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.349200 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.348388 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3cd471ea-70bb-4c9c-b866-674577a98819-podres\") pod \"perf-node-gather-daemonset-plwhb\" (UID: \"3cd471ea-70bb-4c9c-b866-674577a98819\") " pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.349275 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.347870 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cd471ea-70bb-4c9c-b866-674577a98819-lib-modules\") pod \"perf-node-gather-daemonset-plwhb\" (UID: \"3cd471ea-70bb-4c9c-b866-674577a98819\") " pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.357118 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.357085 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ns65\" (UniqueName: \"kubernetes.io/projected/3cd471ea-70bb-4c9c-b866-674577a98819-kube-api-access-7ns65\") pod \"perf-node-gather-daemonset-plwhb\" (UID: \"3cd471ea-70bb-4c9c-b866-674577a98819\") " pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.511129 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.511040 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:50.810243 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:50.810030 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb"] Sep 19 00:18:50.813253 ip-10-0-129-146 kubenswrapper[2568]: W0919 00:18:50.813207 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3cd471ea_70bb_4c9c_b866_674577a98819.slice/crio-cc63faf4a4d10fc01d692e148b6a570f7bfe70da54af26cd4bc6088d54aed9cf WatchSource:0}: Error finding container cc63faf4a4d10fc01d692e148b6a570f7bfe70da54af26cd4bc6088d54aed9cf: Status 404 returned error can't find the container with id cc63faf4a4d10fc01d692e148b6a570f7bfe70da54af26cd4bc6088d54aed9cf Sep 19 00:18:51.597274 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:51.597242 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2zk5g_d8619f0e-fc65-4fbf-9dd2-2c612a94997f/dns/0.log" Sep 19 00:18:51.618481 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:51.618454 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2zk5g_d8619f0e-fc65-4fbf-9dd2-2c612a94997f/kube-rbac-proxy/0.log" Sep 19 00:18:51.719741 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:51.719701 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" event={"ID":"3cd471ea-70bb-4c9c-b866-674577a98819","Type":"ContainerStarted","Data":"ce3ce3e53992c65ed81f800efae50ea689c09b09261f62276c933bb242f69a20"} Sep 19 00:18:51.719918 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:51.719751 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" event={"ID":"3cd471ea-70bb-4c9c-b866-674577a98819","Type":"ContainerStarted","Data":"cc63faf4a4d10fc01d692e148b6a570f7bfe70da54af26cd4bc6088d54aed9cf"} Sep 19 00:18:51.719918 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:51.719849 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:51.736757 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:51.736704 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" podStartSLOduration=1.736688817 podStartE2EDuration="1.736688817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 00:18:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 00:18:51.736039219 +0000 UTC m=+1222.889633084" watchObservedRunningTime="2026-09-19 00:18:51.736688817 +0000 UTC m=+1222.890282682" Sep 19 00:18:51.765409 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:51.765383 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-frlfs_1df5be81-ad93-46f5-9886-90e729bdc767/dns-node-resolver/0.log" Sep 19 00:18:52.308692 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:52.308669 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29829600-s2s2h_30b28e1b-b113-4021-aa68-11e495f07f39/image-pruner/0.log" Sep 19 00:18:52.362887 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:52.362850 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2k8jv_109db1fc-2eed-4932-b461-7acbf67f88e7/node-ca/0.log" Sep 19 00:18:53.307920 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:53.307876 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cq79tq_b1eed8ec-0fec-4500-aec4-788b93b81da7/istio-proxy/0.log" Sep 19 00:18:53.390087 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:53.390055 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-755c79bd44-h9hhm_fcfe7377-8fff-48f2-a4ec-d111f474103f/kube-auth-proxy/0.log" Sep 19 00:18:54.132148 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:54.132121 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j4srb_f1db5e3f-6ee9-4059-bde8-7c257b1f343d/serve-healthcheck-canary/0.log" Sep 19 00:18:54.588311 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:54.588236 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7s9nr_ed90be5c-28d1-4406-982a-e165bf2211e1/kube-rbac-proxy/0.log" Sep 19 00:18:54.608429 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:54.608405 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7s9nr_ed90be5c-28d1-4406-982a-e165bf2211e1/exporter/0.log" Sep 19 00:18:54.630333 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:54.630307 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7s9nr_ed90be5c-28d1-4406-982a-e165bf2211e1/extractor/0.log" Sep 19 00:18:56.898887 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:56.898860 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-ksrzk_2edefa5f-0032-4bc8-a0ad-0fd54ef16e55/server/0.log" Sep 19 00:18:56.968961 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:56.968929 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kwnhz_f2735ad3-d056-44c9-b275-9b7c3b940294/manager/0.log" Sep 19 00:18:56.989732 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:56.989703 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-577c99c85d-5qdtd_68dc1819-e2f2-4c8d-b2a0-71ae6526698a/manager/0.log" Sep 19 00:18:57.011593 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:57.011566 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-577c99c85d-5qdtd_68dc1819-e2f2-4c8d-b2a0-71ae6526698a/copy-manifests/0.log" Sep 19 00:18:57.735462 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:57.735434 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9kf4c/perf-node-gather-daemonset-plwhb" Sep 19 00:18:58.290626 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:18:58.290595 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7968c85699-z2mnc_6237e094-18a6-4593-8525-94a7614f0d22/manager/0.log" Sep 19 00:19:00.261535 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:00.261508 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cq79tq_b1eed8ec-0fec-4500-aec4-788b93b81da7/istio-proxy/0.log" Sep 19 00:19:00.323604 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:00.323570 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-755c79bd44-h9hhm_fcfe7377-8fff-48f2-a4ec-d111f474103f/kube-auth-proxy/0.log" Sep 19 00:19:01.477885 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:01.477818 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cq79tq_b1eed8ec-0fec-4500-aec4-788b93b81da7/istio-proxy/0.log" Sep 19 00:19:01.545380 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:01.545301 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-755c79bd44-h9hhm_fcfe7377-8fff-48f2-a4ec-d111f474103f/kube-auth-proxy/0.log" Sep 19 00:19:03.520515 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:03.520476 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-flxrv_41c42309-992e-4bec-a4bc-9ce6de529b4e/migrator/0.log" Sep 19 00:19:03.539022 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:03.538988 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-flxrv_41c42309-992e-4bec-a4bc-9ce6de529b4e/graceful-termination/0.log" Sep 19 00:19:04.899381 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:04.899353 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-24d8b_6691ea3d-128e-47ee-b52b-2a48ca799dc8/kube-multus-additional-cni-plugins/0.log" Sep 19 00:19:04.918082 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:04.918058 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-24d8b_6691ea3d-128e-47ee-b52b-2a48ca799dc8/egress-router-binary-copy/0.log" Sep 19 00:19:04.936184 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:04.936159 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-24d8b_6691ea3d-128e-47ee-b52b-2a48ca799dc8/cni-plugins/0.log" Sep 19 00:19:04.956480 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:04.956444 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-24d8b_6691ea3d-128e-47ee-b52b-2a48ca799dc8/bond-cni-plugin/0.log" Sep 19 00:19:04.974838 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:04.974812 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-24d8b_6691ea3d-128e-47ee-b52b-2a48ca799dc8/routeoverride-cni/0.log" Sep 19 00:19:04.992941 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:04.992922 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-24d8b_6691ea3d-128e-47ee-b52b-2a48ca799dc8/whereabouts-cni-bincopy/0.log" Sep 19 00:19:05.011362 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:05.011343 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-24d8b_6691ea3d-128e-47ee-b52b-2a48ca799dc8/whereabouts-cni/0.log" Sep 19 00:19:05.430026 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:05.429998 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-wbld8_6523199f-3f99-4e7f-ac81-4462f800b35d/kube-multus/0.log" Sep 19 00:19:05.485054 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:05.485029 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-g2zzv_0092b756-14b9-4468-b25e-47a279c0e9a1/network-metrics-daemon/0.log" Sep 19 00:19:05.502329 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:05.502305 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-g2zzv_0092b756-14b9-4468-b25e-47a279c0e9a1/kube-rbac-proxy/0.log" Sep 19 00:19:06.561532 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:06.561500 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mmfk5_ad0b3adb-7a95-473f-b3c9-2ac12f522495/ovn-controller/0.log" Sep 19 00:19:06.579728 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:06.579695 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mmfk5_ad0b3adb-7a95-473f-b3c9-2ac12f522495/ovn-acl-logging/0.log" Sep 19 00:19:06.602023 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:06.601974 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mmfk5_ad0b3adb-7a95-473f-b3c9-2ac12f522495/kube-rbac-proxy-node/0.log" Sep 19 00:19:06.622312 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:06.622288 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mmfk5_ad0b3adb-7a95-473f-b3c9-2ac12f522495/kube-rbac-proxy-ovn-metrics/0.log" Sep 19 00:19:06.641382 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:06.641346 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mmfk5_ad0b3adb-7a95-473f-b3c9-2ac12f522495/northd/0.log" Sep 19 00:19:06.661226 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:06.660951 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mmfk5_ad0b3adb-7a95-473f-b3c9-2ac12f522495/nbdb/0.log" Sep 19 00:19:06.682232 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:06.682213 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mmfk5_ad0b3adb-7a95-473f-b3c9-2ac12f522495/sbdb/0.log" Sep 19 00:19:06.738668 ip-10-0-129-146 kubenswrapper[2568]: I0919 00:19:06.738641 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mmfk5_ad0b3adb-7a95-473f-b3c9-2ac12f522495/ovnkube-controller/0.log"