Sep 07 12:58:56.156605 ip-10-0-139-45 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 07 12:58:56.156615 ip-10-0-139-45 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 07 12:58:56.156625 ip-10-0-139-45 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 07 12:58:56.156833 ip-10-0-139-45 systemd[1]: Failed to start Kubernetes Kubelet. Sep 07 12:59:06.313549 ip-10-0-139-45 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 07 12:59:06.313569 ip-10-0-139-45 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 4d95e8a47ac3433ca007457df436d415 -- Sep 07 13:00:39.308350 ip-10-0-139-45 systemd[1]: Starting Kubernetes Kubelet... Sep 07 13:00:39.715433 ip-10-0-139-45 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 13:00:39.715433 ip-10-0-139-45 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 07 13:00:39.715433 ip-10-0-139-45 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 13:00:39.715433 ip-10-0-139-45 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 07 13:00:39.715433 ip-10-0-139-45 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 13:00:39.717546 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.717453 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 07 13:00:39.723482 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723465 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 13:00:39.723482 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723483 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723486 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723490 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723492 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723495 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723498 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723500 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723503 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723505 2564 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723508 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723510 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723513 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723516 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723518 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723522 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723524 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723527 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723532 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723536 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723539 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 13:00:39.723548 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723542 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723545 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723547 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723550 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723553 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723556 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723558 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723561 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723564 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723567 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723569 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723572 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723574 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723577 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723580 2564 feature_gate.go:328] unrecognized feature gate: Example Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723582 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723587 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723590 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723593 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 13:00:39.724021 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723596 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723598 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723601 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723604 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723607 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723610 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723612 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723615 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723618 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723620 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723622 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723625 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723627 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723630 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723633 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723636 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723639 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723641 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723644 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723646 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 13:00:39.724489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723649 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723651 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723654 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723656 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723659 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723661 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723664 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723666 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723669 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723671 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723674 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723677 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723679 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723682 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723684 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723687 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723689 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723692 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723695 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 13:00:39.724972 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723697 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723700 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723702 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723705 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723707 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723710 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723712 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723716 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723720 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723722 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723725 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723727 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723730 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723733 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723735 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723738 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723742 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723745 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723748 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 13:00:39.725452 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.723750 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 13:00:39.726260 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726247 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 13:00:39.726260 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726258 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726262 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726265 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726268 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726271 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726274 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726277 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726280 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726282 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726285 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726288 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726290 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726293 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726296 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726298 2564 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726301 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726304 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726306 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726309 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 13:00:39.726316 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726312 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726314 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726317 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726319 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726322 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726325 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726328 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726331 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726334 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726336 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726339 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726341 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726344 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726346 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726350 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726354 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726357 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726359 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726363 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726366 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 13:00:39.726792 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726369 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726371 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726374 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726376 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726379 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726381 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726384 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726386 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726390 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726392 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726396 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726400 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726402 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726405 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726409 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726413 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726417 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726420 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726423 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 13:00:39.727306 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726426 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726429 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726431 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726434 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726437 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726439 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726442 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726445 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726448 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726450 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726453 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726455 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726459 2564 feature_gate.go:328] unrecognized feature gate: Example Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726461 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726464 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726466 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726469 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726471 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726474 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726476 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 13:00:39.727760 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726479 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726482 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726484 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726487 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726489 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726492 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726494 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726497 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726500 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726502 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726505 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726508 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726511 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726513 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726516 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726518 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726521 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726524 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726526 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726529 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 13:00:39.728267 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726898 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726902 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726905 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726908 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726911 2564 feature_gate.go:328] unrecognized feature gate: Example Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726914 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726917 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726920 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726922 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726925 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726928 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726930 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726933 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726936 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726939 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726942 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726944 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726947 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726950 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726952 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 13:00:39.728763 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726955 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726958 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726960 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726963 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726966 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726969 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726971 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726973 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726976 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726979 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726982 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726984 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726986 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726989 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726993 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726996 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.726999 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727003 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727005 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 13:00:39.729274 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727008 2564 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727011 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727013 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727016 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727021 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727025 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727029 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727032 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727034 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727038 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727040 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727043 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727045 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727062 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727067 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727078 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727081 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727084 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727086 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 13:00:39.729721 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727089 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727091 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727094 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727097 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727100 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727102 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727105 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727107 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727110 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727113 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727115 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727118 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727121 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727125 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727128 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727130 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727134 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727138 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727140 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727143 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 13:00:39.730204 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727145 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727148 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727151 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727153 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727155 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727158 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727160 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727163 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727165 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727168 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727170 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727172 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727175 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727177 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727180 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727183 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727185 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727188 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727191 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727194 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 13:00:39.730688 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727196 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727280 2564 flags.go:64] FLAG: --address="0.0.0.0" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727325 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727333 2564 flags.go:64] FLAG: --anonymous-auth="true" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727338 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727343 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727346 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727350 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727355 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727360 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727363 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727367 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727371 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727374 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727377 2564 flags.go:64] FLAG: --cgroup-root="" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727380 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727383 2564 flags.go:64] FLAG: --client-ca-file="" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727386 2564 flags.go:64] FLAG: --cloud-provider="external" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727389 2564 flags.go:64] FLAG: --cluster-dns="[]" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727394 2564 flags.go:64] FLAG: --cluster-domain="" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727397 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727400 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727403 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727406 2564 flags.go:64] FLAG: --container-log-max-files="5" Sep 07 13:00:39.731224 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727410 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727413 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727417 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727420 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727423 2564 flags.go:64] FLAG: --contention-profiling="false" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727426 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727429 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727432 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727436 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727442 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727445 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727449 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727452 2564 flags.go:64] FLAG: --enable-load-reader="false" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727455 2564 flags.go:64] FLAG: --enable-server="true" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727459 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727464 2564 flags.go:64] FLAG: --event-burst="100" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727468 2564 flags.go:64] FLAG: --event-qps="50" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727472 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727475 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727478 2564 flags.go:64] FLAG: --eviction-hard="" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727481 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727484 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727488 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727491 2564 flags.go:64] FLAG: --eviction-soft="" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727494 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 07 13:00:39.731806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727496 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727500 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727502 2564 flags.go:64] FLAG: --experimental-mounter-path="" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727506 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727508 2564 flags.go:64] FLAG: --fail-swap-on="true" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727511 2564 flags.go:64] FLAG: --feature-gates="" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727515 2564 flags.go:64] FLAG: --file-check-frequency="20s" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727519 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727522 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727525 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727528 2564 flags.go:64] FLAG: --healthz-port="10248" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727531 2564 flags.go:64] FLAG: --help="false" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727534 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-139-45.ec2.internal" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727537 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727540 2564 flags.go:64] FLAG: --http-check-frequency="20s" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727543 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727546 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727550 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727552 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727555 2564 flags.go:64] FLAG: --image-service-endpoint="" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727558 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727561 2564 flags.go:64] FLAG: --kube-api-burst="100" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727570 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727573 2564 flags.go:64] FLAG: --kube-api-qps="50" Sep 07 13:00:39.732465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727578 2564 flags.go:64] FLAG: --kube-reserved="" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727581 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727584 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727587 2564 flags.go:64] FLAG: --kubelet-cgroups="" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727590 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727592 2564 flags.go:64] FLAG: --lock-file="" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727595 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727598 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727601 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727606 2564 flags.go:64] FLAG: --log-json-split-stream="false" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727609 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727612 2564 flags.go:64] FLAG: --log-text-split-stream="false" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727615 2564 flags.go:64] FLAG: --logging-format="text" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727621 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727625 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727627 2564 flags.go:64] FLAG: --manifest-url="" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727630 2564 flags.go:64] FLAG: --manifest-url-header="" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727634 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727637 2564 flags.go:64] FLAG: --max-open-files="1000000" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727641 2564 flags.go:64] FLAG: --max-pods="110" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727644 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727649 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727652 2564 flags.go:64] FLAG: --memory-manager-policy="None" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727655 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727657 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 07 13:00:39.733082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727660 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727663 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727670 2564 flags.go:64] FLAG: --node-status-max-images="50" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727673 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727676 2564 flags.go:64] FLAG: --oom-score-adj="-999" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727679 2564 flags.go:64] FLAG: --pod-cidr="" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727688 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727694 2564 flags.go:64] FLAG: --pod-manifest-path="" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727698 2564 flags.go:64] FLAG: --pod-max-pids="-1" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727701 2564 flags.go:64] FLAG: --pods-per-core="0" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727704 2564 flags.go:64] FLAG: --port="10250" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727707 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727710 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07fd3478cdcb07a2c" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727713 2564 flags.go:64] FLAG: --qos-reserved="" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727716 2564 flags.go:64] FLAG: --read-only-port="10255" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727719 2564 flags.go:64] FLAG: --register-node="true" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727722 2564 flags.go:64] FLAG: --register-with-taints="" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727727 2564 flags.go:64] FLAG: --registry-burst="10" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727730 2564 flags.go:64] FLAG: --registry-qps="5" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727733 2564 flags.go:64] FLAG: --reserved-cpus="" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727736 2564 flags.go:64] FLAG: --reserved-memory="" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727740 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727743 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727746 2564 flags.go:64] FLAG: --rotate-certificates="false" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727748 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 07 13:00:39.733706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727751 2564 flags.go:64] FLAG: --runonce="false" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727754 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727757 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727761 2564 flags.go:64] FLAG: --seccomp-default="false" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727764 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727768 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727771 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727774 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727777 2564 flags.go:64] FLAG: --storage-driver-password="root" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727780 2564 flags.go:64] FLAG: --storage-driver-secure="false" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727783 2564 flags.go:64] FLAG: --storage-driver-table="stats" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727786 2564 flags.go:64] FLAG: --storage-driver-user="root" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727788 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727791 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727800 2564 flags.go:64] FLAG: --system-cgroups="" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727804 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727810 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727813 2564 flags.go:64] FLAG: --tls-cert-file="" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727815 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727819 2564 flags.go:64] FLAG: --tls-min-version="" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727822 2564 flags.go:64] FLAG: --tls-private-key-file="" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727825 2564 flags.go:64] FLAG: --topology-manager-policy="none" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727828 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727831 2564 flags.go:64] FLAG: --topology-manager-scope="container" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727833 2564 flags.go:64] FLAG: --v="2" Sep 07 13:00:39.734330 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727837 2564 flags.go:64] FLAG: --version="false" Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727841 2564 flags.go:64] FLAG: --vmodule="" Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727845 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.727848 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727964 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727968 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727971 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727974 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727977 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727980 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727984 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727987 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727990 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727992 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727995 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.727997 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728000 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728002 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728005 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728007 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728010 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 13:00:39.734986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728013 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728021 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728024 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728026 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728029 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728031 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728034 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728038 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728042 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728045 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728064 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728067 2564 feature_gate.go:328] unrecognized feature gate: Example Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728070 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728073 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728075 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728079 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728083 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728086 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728089 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 13:00:39.735532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728091 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728094 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728098 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728101 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728104 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728107 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728109 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728112 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728115 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728117 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728120 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728122 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728125 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728127 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728132 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728135 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728138 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728140 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728143 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728145 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 13:00:39.735986 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728148 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728150 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728153 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728155 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728158 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728160 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728163 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728165 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728168 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728171 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728174 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728176 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728179 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728181 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728185 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728188 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728190 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728193 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728195 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 13:00:39.736492 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728198 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728201 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728203 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728206 2564 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728208 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728210 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728213 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728217 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728219 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728223 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728225 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728229 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728232 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728236 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728239 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728241 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728244 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728247 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728250 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 13:00:39.736957 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728252 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 13:00:39.737437 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728255 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 13:00:39.737437 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728258 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 13:00:39.737437 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728260 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 13:00:39.737437 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.728263 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 13:00:39.737437 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.729064 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 13:00:39.737437 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.737067 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 07 13:00:39.737437 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.737083 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 07 13:00:39.738011 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.737999 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 07 13:00:39.738045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.738010 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738081 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738087 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738091 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738094 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738097 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738101 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738104 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738107 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738110 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738113 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738117 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738120 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738122 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738125 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738128 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738131 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738134 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738136 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738139 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 13:00:39.738189 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738141 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738144 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738147 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738149 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738154 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738157 2564 feature_gate.go:328] unrecognized feature gate: Example Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738160 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738163 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738166 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738169 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738173 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738175 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738178 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738180 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738184 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738187 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738190 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738193 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738196 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738199 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 13:00:39.738707 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738202 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738204 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738207 2564 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738210 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738213 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738215 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738218 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738221 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738223 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738226 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738229 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738231 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738235 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738238 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738242 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738244 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738247 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738251 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738256 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 13:00:39.739218 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738259 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738262 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738265 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738268 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738271 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738273 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738276 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738280 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738283 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738286 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738288 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738291 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738294 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738296 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738299 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738301 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738304 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738307 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738309 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 13:00:39.739674 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738312 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738314 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738317 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738319 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738322 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738325 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738328 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738331 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738334 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738336 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738339 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738341 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738344 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738347 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738350 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738352 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738355 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738357 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738360 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738363 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 13:00:39.740153 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738366 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 13:00:39.740924 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738369 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 13:00:39.740924 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.738374 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 13:00:39.740924 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738488 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 13:00:39.740924 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738493 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 13:00:39.740924 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738496 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 13:00:39.740924 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738499 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 13:00:39.740924 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738502 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 13:00:39.740924 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738505 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 13:00:39.740924 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738507 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 13:00:39.740924 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738512 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 13:00:39.740924 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738515 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 13:00:39.740924 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738518 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 13:00:39.740924 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738522 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 13:00:39.740924 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738526 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738529 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738532 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738535 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738537 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738541 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738545 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738548 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738550 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738553 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738556 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738559 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738561 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738564 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738567 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738569 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738572 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738575 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738578 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 13:00:39.741514 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738581 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738583 2564 feature_gate.go:328] unrecognized feature gate: Example Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738586 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738589 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738591 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738594 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738597 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738599 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738602 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738605 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738607 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738610 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738612 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738615 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738617 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738620 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738622 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738625 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738628 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738631 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 13:00:39.741976 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738633 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738635 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738638 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738641 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738643 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738646 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738648 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738651 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738653 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738656 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738658 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738661 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738664 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738667 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738669 2564 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738672 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738674 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738677 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738680 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738682 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 13:00:39.742477 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738685 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738687 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738690 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738692 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738695 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738697 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738700 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738702 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738705 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738708 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738711 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738713 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738716 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738719 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738722 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738725 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738728 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738730 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738733 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738735 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 13:00:39.742977 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738738 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 13:00:39.743478 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738740 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 13:00:39.743478 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738743 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 13:00:39.743478 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738745 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 13:00:39.743478 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738748 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 13:00:39.743478 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738751 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 13:00:39.743478 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738754 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 13:00:39.743478 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738756 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 13:00:39.743478 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:39.738759 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 13:00:39.743478 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.738764 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 13:00:39.743478 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.738886 2564 server.go:962] "Client rotation is on, will bootstrap in background" Sep 07 13:00:39.743748 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.743666 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 07 13:00:39.744806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.744794 2564 server.go:1019] "Starting client certificate rotation" Sep 07 13:00:39.744905 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.744888 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 07 13:00:39.744934 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.744927 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 07 13:00:39.747704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.747692 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 07 13:00:39.751105 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.751082 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 07 13:00:39.763843 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.763796 2564 log.go:25] "Validated CRI v1 runtime API" Sep 07 13:00:39.767854 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.767837 2564 log.go:25] "Validated CRI v1 image API" Sep 07 13:00:39.770375 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.770358 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 07 13:00:39.773639 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.773621 2564 fs.go:135] Filesystem UUIDs: map[2a2c5e7f-5efe-4405-aceb-9eb11d947a3b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f4c3bcb8-2a44-4290-a464-8cbefc108b9e:/dev/nvme0n1p4] Sep 07 13:00:39.773687 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.773640 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 07 13:00:39.778288 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.778272 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 07 13:00:39.779033 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.778922 2564 manager.go:216] Machine: {Timestamp:2026-09-07 13:00:39.777371541 +0000 UTC m=+0.372876148 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103410 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d70e2291271dcdf9673834316cafd SystemUUID:ec2d70e2-2912-71dc-df96-73834316cafd BootID:4d95e8a4-7ac3-433c-a007-457df436d415 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c9:b5:34:d5:ed Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c9:b5:34:d5:ed Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:af:a7:47:fc:cf Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 07 13:00:39.779033 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.779031 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 07 13:00:39.779157 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.779138 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 07 13:00:39.782789 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.782765 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 07 13:00:39.782943 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.782793 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-45.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 07 13:00:39.783657 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.783647 2564 topology_manager.go:138] "Creating topology manager with none policy" Sep 07 13:00:39.783698 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.783659 2564 container_manager_linux.go:309] "Creating device plugin manager" Sep 07 13:00:39.783698 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.783667 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 07 13:00:39.784540 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.784529 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 07 13:00:39.785774 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.785764 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 07 13:00:39.791097 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.791084 2564 state_mem.go:36] "Initialized new in-memory state store" Sep 07 13:00:39.791965 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.791912 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 07 13:00:39.792080 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.792008 2564 kubelet.go:486] "Attempting to sync node with API server" Sep 07 13:00:39.792080 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.792030 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 07 13:00:39.792080 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.792068 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 07 13:00:39.792080 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.792082 2564 kubelet.go:398] "Adding apiserver pod source" Sep 07 13:00:39.792254 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.792094 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 07 13:00:39.792608 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.792588 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 07 13:00:39.792670 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.792622 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 07 13:00:39.795673 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.795650 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 07 13:00:39.798659 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.798646 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 07 13:00:39.798711 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.798676 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 07 13:00:39.800093 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.800082 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 07 13:00:39.800128 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.800097 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 07 13:00:39.800128 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.800105 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 07 13:00:39.800128 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.800110 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 07 13:00:39.800128 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.800120 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 07 13:00:39.800128 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.800125 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 07 13:00:39.800259 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.800139 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 07 13:00:39.800259 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.800146 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 07 13:00:39.800259 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.800152 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 07 13:00:39.800259 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.800158 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 07 13:00:39.800259 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.800166 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 07 13:00:39.800259 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.800171 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 07 13:00:39.801030 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.801019 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 07 13:00:39.801030 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.801030 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 07 13:00:39.803896 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:39.803870 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 07 13:00:39.803970 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:39.803915 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-45.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 07 13:00:39.803970 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.803938 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-45.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 13:00:39.805206 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.805194 2564 server.go:1268] "Started kubelet" Sep 07 13:00:39.805319 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.805271 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 07 13:00:39.805354 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.805339 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 07 13:00:39.805685 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.805267 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 07 13:00:39.806046 ip-10-0-139-45 systemd[1]: Started Kubernetes Kubelet. Sep 07 13:00:39.807146 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.806768 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 07 13:00:39.807702 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.807639 2564 server.go:310] "Adding debug handlers to kubelet server" Sep 07 13:00:39.809480 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:39.809460 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 07 13:00:39.813789 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.813769 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 07 13:00:39.814481 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.814461 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 07 13:00:39.814595 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:39.814581 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-45.ec2.internal\" not found" Sep 07 13:00:39.815327 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.815310 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 07 13:00:39.815327 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.815322 2564 factory.go:55] Registering systemd factory Sep 07 13:00:39.815327 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.815330 2564 factory.go:223] Registration of the systemd container factory successfully Sep 07 13:00:39.815519 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.815433 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 07 13:00:39.815519 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.815449 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 07 13:00:39.815597 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.815554 2564 factory.go:153] Registering CRI-O factory Sep 07 13:00:39.815597 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.815563 2564 reconstruct.go:97] "Volume reconstruction finished" Sep 07 13:00:39.815597 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.815567 2564 factory.go:223] Registration of the crio container factory successfully Sep 07 13:00:39.815597 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.815571 2564 reconciler.go:29] "Reconciler: start to sync state" Sep 07 13:00:39.815597 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.815582 2564 factory.go:103] Registering Raw factory Sep 07 13:00:39.815597 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.815600 2564 manager.go:1196] Started watching for new ooms in manager Sep 07 13:00:39.816034 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.816014 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 07 13:00:39.816110 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.816092 2564 manager.go:349] Starting recovery of all containers Sep 07 13:00:39.822237 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:39.818743 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-45.ec2.internal.18d30b6d01c3cc9e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-45.ec2.internal,UID:ip-10-0-139-45.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-45.ec2.internal,},FirstTimestamp:2026-09-07 13:00:39.805168798 +0000 UTC m=+0.400673406,LastTimestamp:2026-09-07 13:00:39.805168798 +0000 UTC m=+0.400673406,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-45.ec2.internal,}" Sep 07 13:00:39.822336 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:39.822265 2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-45.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 07 13:00:39.823344 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:39.823314 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 07 13:00:39.829837 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.829816 2564 manager.go:354] Recovery completed Sep 07 13:00:39.834394 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.834380 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 13:00:39.836384 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.836368 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasSufficientMemory" Sep 07 13:00:39.836462 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.836400 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 13:00:39.836462 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.836410 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasSufficientPID" Sep 07 13:00:39.836986 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.836967 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 07 13:00:39.837045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.836987 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 07 13:00:39.837045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.837003 2564 state_mem.go:36] "Initialized new in-memory state store" Sep 07 13:00:39.837147 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.837002 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 13:00:39.837655 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.837636 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k67mp" Sep 07 13:00:39.837933 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.837918 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasSufficientMemory" Sep 07 13:00:39.837975 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.837946 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 13:00:39.837975 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.837956 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasSufficientPID" Sep 07 13:00:39.838698 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:39.838633 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-45.ec2.internal.18d30b6d03a0222a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-45.ec2.internal,UID:ip-10-0-139-45.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-45.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-45.ec2.internal,},FirstTimestamp:2026-09-07 13:00:39.836385834 +0000 UTC m=+0.431890441,LastTimestamp:2026-09-07 13:00:39.836385834 +0000 UTC m=+0.431890441,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-45.ec2.internal,}" Sep 07 13:00:39.839192 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.839179 2564 policy_none.go:49] "None policy: Start" Sep 07 13:00:39.839266 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.839198 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 07 13:00:39.839266 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.839212 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 07 13:00:39.841276 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.841262 2564 policy_none.go:47] "Start" Sep 07 13:00:39.849222 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.849111 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k67mp" Sep 07 13:00:39.849366 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:39.849299 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-45.ec2.internal.18d30b6d03a06ba7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-45.ec2.internal,UID:ip-10-0-139-45.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-45.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-45.ec2.internal,},FirstTimestamp:2026-09-07 13:00:39.836404647 +0000 UTC m=+0.431909254,LastTimestamp:2026-09-07 13:00:39.836404647 +0000 UTC m=+0.431909254,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-45.ec2.internal,}" Sep 07 13:00:39.871678 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.871662 2564 manager.go:337] "Starting Device Plugin manager" Sep 07 13:00:39.871794 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:39.871705 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 07 13:00:39.871794 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.871718 2564 server.go:87] "Starting device plugin registration server" Sep 07 13:00:39.872038 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.872025 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 07 13:00:39.872114 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.872042 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 07 13:00:39.872673 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.872225 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 07 13:00:39.872673 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.872299 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 07 13:00:39.872673 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.872307 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 07 13:00:39.872798 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:39.872743 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 07 13:00:39.872798 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:39.872792 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-45.ec2.internal\" not found" Sep 07 13:00:39.914746 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.914704 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 07 13:00:39.916986 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.915964 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 07 13:00:39.916986 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.916008 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 07 13:00:39.916986 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.916034 2564 kubelet.go:2448] "Starting kubelet main sync loop" Sep 07 13:00:39.916986 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:39.916148 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 07 13:00:39.918327 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.918308 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 13:00:39.973170 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.973038 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 13:00:39.974139 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.974121 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasSufficientMemory" Sep 07 13:00:39.974235 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.974154 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 13:00:39.974235 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.974173 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasSufficientPID" Sep 07 13:00:39.974235 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.974201 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-45.ec2.internal" Sep 07 13:00:39.982149 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:39.982132 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-45.ec2.internal" Sep 07 13:00:39.982226 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:39.982156 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-45.ec2.internal\": node \"ip-10-0-139-45.ec2.internal\" not found" Sep 07 13:00:40.010425 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:40.010396 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-45.ec2.internal\" not found" Sep 07 13:00:40.016536 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.016498 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-45.ec2.internal"] Sep 07 13:00:40.016633 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.016602 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 13:00:40.018079 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.018061 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasSufficientMemory" Sep 07 13:00:40.018175 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.018096 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 13:00:40.018175 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.018110 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasSufficientPID" Sep 07 13:00:40.020607 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.020590 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 13:00:40.020766 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.020751 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal" Sep 07 13:00:40.020818 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.020780 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 13:00:40.021465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.021449 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasSufficientMemory" Sep 07 13:00:40.021545 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.021476 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasSufficientMemory" Sep 07 13:00:40.021545 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.021487 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 13:00:40.021545 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.021502 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasSufficientPID" Sep 07 13:00:40.021695 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.021502 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 13:00:40.021695 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.021586 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasSufficientPID" Sep 07 13:00:40.023778 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.023763 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-45.ec2.internal" Sep 07 13:00:40.023863 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.023788 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 13:00:40.024560 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.024535 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasSufficientMemory" Sep 07 13:00:40.024646 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.024567 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 13:00:40.024646 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.024583 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeHasSufficientPID" Sep 07 13:00:40.048803 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:40.048783 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-45.ec2.internal\" not found" node="ip-10-0-139-45.ec2.internal" Sep 07 13:00:40.053111 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:40.053096 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-45.ec2.internal\" not found" node="ip-10-0-139-45.ec2.internal" Sep 07 13:00:40.110765 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:40.110734 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-45.ec2.internal\" not found" Sep 07 13:00:40.211241 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:40.211202 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-45.ec2.internal\" not found" Sep 07 13:00:40.217571 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.217544 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d905437f79aa56bfe462c09feb96e548-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal\" (UID: \"d905437f79aa56bfe462c09feb96e548\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal" Sep 07 13:00:40.217645 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.217575 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e4dcf97b5b5636889d8eabaae498c349-config\") pod \"kube-apiserver-proxy-ip-10-0-139-45.ec2.internal\" (UID: \"e4dcf97b5b5636889d8eabaae498c349\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-45.ec2.internal" Sep 07 13:00:40.217645 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.217592 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d905437f79aa56bfe462c09feb96e548-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal\" (UID: \"d905437f79aa56bfe462c09feb96e548\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal" Sep 07 13:00:40.312098 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:40.312025 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-45.ec2.internal\" not found" Sep 07 13:00:40.318377 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.318343 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d905437f79aa56bfe462c09feb96e548-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal\" (UID: \"d905437f79aa56bfe462c09feb96e548\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal" Sep 07 13:00:40.318442 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.318383 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d905437f79aa56bfe462c09feb96e548-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal\" (UID: \"d905437f79aa56bfe462c09feb96e548\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal" Sep 07 13:00:40.318442 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.318400 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e4dcf97b5b5636889d8eabaae498c349-config\") pod \"kube-apiserver-proxy-ip-10-0-139-45.ec2.internal\" (UID: \"e4dcf97b5b5636889d8eabaae498c349\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-45.ec2.internal" Sep 07 13:00:40.318442 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.318430 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e4dcf97b5b5636889d8eabaae498c349-config\") pod \"kube-apiserver-proxy-ip-10-0-139-45.ec2.internal\" (UID: \"e4dcf97b5b5636889d8eabaae498c349\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-45.ec2.internal" Sep 07 13:00:40.318543 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.318447 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d905437f79aa56bfe462c09feb96e548-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal\" (UID: \"d905437f79aa56bfe462c09feb96e548\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal" Sep 07 13:00:40.318543 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.318447 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d905437f79aa56bfe462c09feb96e548-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal\" (UID: \"d905437f79aa56bfe462c09feb96e548\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal" Sep 07 13:00:40.350528 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.350488 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal" Sep 07 13:00:40.356402 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.356376 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-45.ec2.internal" Sep 07 13:00:40.412630 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:40.412589 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-45.ec2.internal\" not found" Sep 07 13:00:40.520453 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:40.520404 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-45.ec2.internal\" not found" Sep 07 13:00:40.621443 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:40.621364 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-45.ec2.internal\" not found" Sep 07 13:00:40.722218 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:40.722177 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-45.ec2.internal\" not found" Sep 07 13:00:40.745115 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.745083 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 07 13:00:40.745269 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.745248 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 07 13:00:40.823021 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:40.822989 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-45.ec2.internal\" not found" Sep 07 13:00:40.855358 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.855315 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-06 12:55:40 +0000 UTC" deadline="2028-03-23 17:49:40.952321509 +0000 UTC" Sep 07 13:00:40.855358 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.855352 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13516h49m0.096973408s" Sep 07 13:00:40.923827 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:40.923744 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-45.ec2.internal\" not found" Sep 07 13:00:40.948815 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:40.948791 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 13:00:41.015009 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.014981 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal" Sep 07 13:00:41.024878 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.024838 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 13:00:41.029322 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.029296 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 07 13:00:41.031380 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.031353 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-45.ec2.internal" Sep 07 13:00:41.044062 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.044038 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 07 13:00:41.166432 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:41.166368 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd905437f79aa56bfe462c09feb96e548.slice/crio-41a1f0bb9f1f546e570c2714bef1490da6630c19300f6924a2166410f91051b4 WatchSource:0}: Error finding container 41a1f0bb9f1f546e570c2714bef1490da6630c19300f6924a2166410f91051b4: Status 404 returned error can't find the container with id 41a1f0bb9f1f546e570c2714bef1490da6630c19300f6924a2166410f91051b4 Sep 07 13:00:41.167151 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.167040 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 13:00:41.169973 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.169955 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 13:00:41.430794 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:41.430762 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4dcf97b5b5636889d8eabaae498c349.slice/crio-2f8e5fa40357a6309d6b927de312725c7c79467008541ae3dc5f303c43da4862 WatchSource:0}: Error finding container 2f8e5fa40357a6309d6b927de312725c7c79467008541ae3dc5f303c43da4862: Status 404 returned error can't find the container with id 2f8e5fa40357a6309d6b927de312725c7c79467008541ae3dc5f303c43da4862 Sep 07 13:00:41.796956 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.796864 2564 apiserver.go:52] "Watching apiserver" Sep 07 13:00:41.804043 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.804018 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 07 13:00:41.804421 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.804399 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-f5cpd","openshift-network-diagnostics/network-check-target-plvr8","openshift-network-operator/iptables-alerter-w27xg","openshift-ovn-kubernetes/ovnkube-node-wxbzw","kube-system/global-pull-secret-syncer-k8jtg","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh","openshift-dns/node-resolver-fcd4k","openshift-image-registry/node-ca-hrgm6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal","openshift-multus/multus-8rdps","kube-system/konnectivity-agent-tj9nj","kube-system/kube-apiserver-proxy-ip-10-0-139-45.ec2.internal","openshift-cluster-node-tuning-operator/tuned-9rrnt","openshift-multus/multus-additional-cni-plugins-4p6gh"] Sep 07 13:00:41.807424 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.807395 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fcd4k" Sep 07 13:00:41.809586 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.809566 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:41.809696 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:41.809662 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:00:41.810159 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.810143 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 07 13:00:41.810408 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.810390 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 07 13:00:41.810491 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.810406 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gmtn7\"" Sep 07 13:00:41.811948 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.811928 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w27xg" Sep 07 13:00:41.814033 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.814015 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 07 13:00:41.814234 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.814210 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:41.814348 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:41.814292 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:00:41.814470 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.814395 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rxq92\"" Sep 07 13:00:41.814470 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.814430 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 07 13:00:41.814739 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.814723 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 07 13:00:41.814787 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.814745 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 07 13:00:41.816722 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.816697 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.818688 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.818673 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 07 13:00:41.818766 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.818703 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pp48v\"" Sep 07 13:00:41.818870 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.818855 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 07 13:00:41.819292 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.819274 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hrgm6" Sep 07 13:00:41.821560 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.821540 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 07 13:00:41.821661 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.821554 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 07 13:00:41.821661 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.821625 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gdd4l\"" Sep 07 13:00:41.821661 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.821641 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 07 13:00:41.822030 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.822014 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.824509 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.824490 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:41.825107 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.825088 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 07 13:00:41.825515 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.825499 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 07 13:00:41.826038 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.825880 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qbm8k\"" Sep 07 13:00:41.826038 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.825897 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 07 13:00:41.826449 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.826129 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 07 13:00:41.826449 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.826176 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 07 13:00:41.826449 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.826180 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 07 13:00:41.828615 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.828487 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 07 13:00:41.828615 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.828499 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 07 13:00:41.828615 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.828510 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 07 13:00:41.828615 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.828553 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vhgqk\"" Sep 07 13:00:41.829075 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829060 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-systemd\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.829130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829086 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npssx\" (UniqueName: \"kubernetes.io/projected/8ed5b1e0-eb99-433d-9df9-43019c49d31a-kube-api-access-npssx\") pod \"iptables-alerter-w27xg\" (UID: \"8ed5b1e0-eb99-433d-9df9-43019c49d31a\") " pod="openshift-network-operator/iptables-alerter-w27xg" Sep 07 13:00:41.829130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829104 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-modprobe-d\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.829219 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829168 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-kubernetes\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.829219 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829198 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-host\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.829219 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829215 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/912823fa-6c24-493a-8640-89b921cdf2f2-hosts-file\") pod \"node-resolver-fcd4k\" (UID: \"912823fa-6c24-493a-8640-89b921cdf2f2\") " pod="openshift-dns/node-resolver-fcd4k" Sep 07 13:00:41.829348 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829230 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/912823fa-6c24-493a-8640-89b921cdf2f2-tmp-dir\") pod \"node-resolver-fcd4k\" (UID: \"912823fa-6c24-493a-8640-89b921cdf2f2\") " pod="openshift-dns/node-resolver-fcd4k" Sep 07 13:00:41.829348 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829248 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdwtp\" (UniqueName: \"kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp\") pod \"network-check-target-plvr8\" (UID: \"e754de95-041a-497a-bee1-8ca095b326f4\") " pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:41.829348 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829280 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-sysctl-d\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.829348 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829306 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.829530 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829310 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-sys\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.829530 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829373 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-var-lib-kubelet\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.829530 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829388 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-tuned\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.829530 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829402 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ed5b1e0-eb99-433d-9df9-43019c49d31a-host-slash\") pod \"iptables-alerter-w27xg\" (UID: \"8ed5b1e0-eb99-433d-9df9-43019c49d31a\") " pod="openshift-network-operator/iptables-alerter-w27xg" Sep 07 13:00:41.829530 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829416 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-sysconfig\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.829530 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829430 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-sysctl-conf\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.829530 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829443 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-run\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.829530 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829468 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-lib-modules\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.829845 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829526 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a85534bb-a70d-4c02-a132-b56a2faf2c47-tmp\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.829845 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829586 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htj7j\" (UniqueName: \"kubernetes.io/projected/a85534bb-a70d-4c02-a132-b56a2faf2c47-kube-api-access-htj7j\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.829845 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829621 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsrb9\" (UniqueName: \"kubernetes.io/projected/912823fa-6c24-493a-8640-89b921cdf2f2-kube-api-access-xsrb9\") pod \"node-resolver-fcd4k\" (UID: \"912823fa-6c24-493a-8640-89b921cdf2f2\") " pod="openshift-dns/node-resolver-fcd4k" Sep 07 13:00:41.829845 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829647 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8ed5b1e0-eb99-433d-9df9-43019c49d31a-iptables-alerter-script\") pod \"iptables-alerter-w27xg\" (UID: \"8ed5b1e0-eb99-433d-9df9-43019c49d31a\") " pod="openshift-network-operator/iptables-alerter-w27xg" Sep 07 13:00:41.829845 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829666 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs\") pod \"network-metrics-daemon-f5cpd\" (UID: \"5fbd07ff-dbb9-46f0-904d-d21419e31373\") " pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:41.829845 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.829693 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btxvg\" (UniqueName: \"kubernetes.io/projected/5fbd07ff-dbb9-46f0-904d-d21419e31373-kube-api-access-btxvg\") pod \"network-metrics-daemon-f5cpd\" (UID: \"5fbd07ff-dbb9-46f0-904d-d21419e31373\") " pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:41.831844 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.831820 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 07 13:00:41.831954 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.831943 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9rcvp\"" Sep 07 13:00:41.832026 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.831972 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tj9nj" Sep 07 13:00:41.832093 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.832051 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 07 13:00:41.832345 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.832171 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 07 13:00:41.832345 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.832242 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 07 13:00:41.833598 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.833574 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 07 13:00:41.834303 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.834287 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 07 13:00:41.834387 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.834320 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 07 13:00:41.834479 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.834458 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-llrfv\"" Sep 07 13:00:41.834643 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.834624 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:41.834733 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:41.834709 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:00:41.837287 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.837262 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:41.839547 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.839526 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-76t9d\"" Sep 07 13:00:41.839649 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.839554 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 07 13:00:41.839649 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.839616 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 07 13:00:41.856055 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.856041 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-glm99" Sep 07 13:00:41.865549 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.865527 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-glm99" Sep 07 13:00:41.917693 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.917662 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 07 13:00:41.921272 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.921215 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-45.ec2.internal" event={"ID":"e4dcf97b5b5636889d8eabaae498c349","Type":"ContainerStarted","Data":"2f8e5fa40357a6309d6b927de312725c7c79467008541ae3dc5f303c43da4862"} Sep 07 13:00:41.923441 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.923407 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal" event={"ID":"d905437f79aa56bfe462c09feb96e548","Type":"ContainerStarted","Data":"41a1f0bb9f1f546e570c2714bef1490da6630c19300f6924a2166410f91051b4"} Sep 07 13:00:41.929960 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.929936 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-modprobe-d\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.930071 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.929966 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/912823fa-6c24-493a-8640-89b921cdf2f2-hosts-file\") pod \"node-resolver-fcd4k\" (UID: \"912823fa-6c24-493a-8640-89b921cdf2f2\") " pod="openshift-dns/node-resolver-fcd4k" Sep 07 13:00:41.930071 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.929983 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/912823fa-6c24-493a-8640-89b921cdf2f2-tmp-dir\") pod \"node-resolver-fcd4k\" (UID: \"912823fa-6c24-493a-8640-89b921cdf2f2\") " pod="openshift-dns/node-resolver-fcd4k" Sep 07 13:00:41.930071 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930002 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-system-cni-dir\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:41.930071 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930030 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3c09e344-284e-410e-8829-3b0bf6d35c93-konnectivity-ca\") pod \"konnectivity-agent-tj9nj\" (UID: \"3c09e344-284e-410e-8829-3b0bf6d35c93\") " pod="kube-system/konnectivity-agent-tj9nj" Sep 07 13:00:41.930071 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930052 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-socket-dir\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:41.930358 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930105 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-var-lib-kubelet\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.930358 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930106 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/912823fa-6c24-493a-8640-89b921cdf2f2-hosts-file\") pod \"node-resolver-fcd4k\" (UID: \"912823fa-6c24-493a-8640-89b921cdf2f2\") " pod="openshift-dns/node-resolver-fcd4k" Sep 07 13:00:41.930358 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930151 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-sysconfig\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.930358 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930185 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-lib-modules\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.930358 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930210 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a85534bb-a70d-4c02-a132-b56a2faf2c47-tmp\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.930358 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930214 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-sysconfig\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.930358 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930254 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-run-netns\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.930358 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930265 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-modprobe-d\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.930358 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930312 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8ed5b1e0-eb99-433d-9df9-43019c49d31a-iptables-alerter-script\") pod \"iptables-alerter-w27xg\" (UID: \"8ed5b1e0-eb99-433d-9df9-43019c49d31a\") " pod="openshift-network-operator/iptables-alerter-w27xg" Sep 07 13:00:41.930358 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930322 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-lib-modules\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.930358 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930338 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-multus-cni-dir\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930377 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/912823fa-6c24-493a-8640-89b921cdf2f2-tmp-dir\") pod \"node-resolver-fcd4k\" (UID: \"912823fa-6c24-493a-8640-89b921cdf2f2\") " pod="openshift-dns/node-resolver-fcd4k" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930365 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-cnibin\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930444 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-hostroot\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930474 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-kubelet-config\") pod \"global-pull-secret-syncer-k8jtg\" (UID: \"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8\") " pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930504 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-kubernetes\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930529 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-run-openvswitch\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930577 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-run-ovn\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930624 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-cni-netd\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930642 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-kubernetes\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930652 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0fc4604b-fd16-466e-8a28-e77878ba81cd-ovn-node-metrics-cert\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930671 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930692 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-run-multus-certs\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930722 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-sysctl-d\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930748 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-sys\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930773 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-var-lib-kubelet\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930799 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:41.930869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930824 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-kubelet\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930848 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0fc4604b-fd16-466e-8a28-e77878ba81cd-ovnkube-config\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930873 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930901 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ed5b1e0-eb99-433d-9df9-43019c49d31a-host-slash\") pod \"iptables-alerter-w27xg\" (UID: \"8ed5b1e0-eb99-433d-9df9-43019c49d31a\") " pod="openshift-network-operator/iptables-alerter-w27xg" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930919 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8ed5b1e0-eb99-433d-9df9-43019c49d31a-iptables-alerter-script\") pod \"iptables-alerter-w27xg\" (UID: \"8ed5b1e0-eb99-433d-9df9-43019c49d31a\") " pod="openshift-network-operator/iptables-alerter-w27xg" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930927 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-run\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930954 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.930978 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-var-lib-openvswitch\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931005 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkxfm\" (UniqueName: \"kubernetes.io/projected/1b2a29b1-b954-4434-99d2-d8168c756c9f-kube-api-access-jkxfm\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931029 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ec3637df-91b4-4cbe-8d8b-7378b2c12a4f-serviceca\") pod \"node-ca-hrgm6\" (UID: \"ec3637df-91b4-4cbe-8d8b-7378b2c12a4f\") " pod="openshift-image-registry/node-ca-hrgm6" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931054 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs\") pod \"network-metrics-daemon-f5cpd\" (UID: \"5fbd07ff-dbb9-46f0-904d-d21419e31373\") " pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931067 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-sysctl-d\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931100 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-systemd\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931106 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-sys\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931143 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-var-lib-kubelet\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931164 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931181 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-systemd\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.931550 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931194 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-var-lib-cni-multus\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.932338 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931212 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-run\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.932338 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931223 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-multus-conf-dir\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.932338 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931236 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8ed5b1e0-eb99-433d-9df9-43019c49d31a-host-slash\") pod \"iptables-alerter-w27xg\" (UID: \"8ed5b1e0-eb99-433d-9df9-43019c49d31a\") " pod="openshift-network-operator/iptables-alerter-w27xg" Sep 07 13:00:41.932338 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931246 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmglh\" (UniqueName: \"kubernetes.io/projected/ec3637df-91b4-4cbe-8d8b-7378b2c12a4f-kube-api-access-vmglh\") pod \"node-ca-hrgm6\" (UID: \"ec3637df-91b4-4cbe-8d8b-7378b2c12a4f\") " pod="openshift-image-registry/node-ca-hrgm6" Sep 07 13:00:41.932338 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:41.931266 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 13:00:41.932338 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931273 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npssx\" (UniqueName: \"kubernetes.io/projected/8ed5b1e0-eb99-433d-9df9-43019c49d31a-kube-api-access-npssx\") pod \"iptables-alerter-w27xg\" (UID: \"8ed5b1e0-eb99-433d-9df9-43019c49d31a\") " pod="openshift-network-operator/iptables-alerter-w27xg" Sep 07 13:00:41.932338 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931297 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-sys-fs\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:41.932338 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931323 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/84352554-8f00-4dfe-8a59-c14dc986c2dd-cni-binary-copy\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.932338 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:41.931347 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs podName:5fbd07ff-dbb9-46f0-904d-d21419e31373 nodeName:}" failed. No retries permitted until 2026-09-07 13:00:42.431325605 +0000 UTC m=+3.026830222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs") pod "network-metrics-daemon-f5cpd" (UID: "5fbd07ff-dbb9-46f0-904d-d21419e31373") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 13:00:41.932338 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931457 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-etc-kubernetes\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.932338 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931509 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-cnibin\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:41.932338 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931533 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-device-dir\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:41.932338 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931570 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jplhk\" (UniqueName: \"kubernetes.io/projected/84352554-8f00-4dfe-8a59-c14dc986c2dd-kube-api-access-jplhk\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.932338 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931594 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret\") pod \"global-pull-secret-syncer-k8jtg\" (UID: \"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8\") " pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:41.932338 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931655 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-sysctl-conf\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.932338 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931687 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htj7j\" (UniqueName: \"kubernetes.io/projected/a85534bb-a70d-4c02-a132-b56a2faf2c47-kube-api-access-htj7j\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931711 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsrb9\" (UniqueName: \"kubernetes.io/projected/912823fa-6c24-493a-8640-89b921cdf2f2-kube-api-access-xsrb9\") pod \"node-resolver-fcd4k\" (UID: \"912823fa-6c24-493a-8640-89b921cdf2f2\") " pod="openshift-dns/node-resolver-fcd4k" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931737 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-node-log\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931749 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-sysctl-conf\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931765 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/84352554-8f00-4dfe-8a59-c14dc986c2dd-multus-daemon-config\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931794 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3c09e344-284e-410e-8829-3b0bf6d35c93-agent-certs\") pod \"konnectivity-agent-tj9nj\" (UID: \"3c09e344-284e-410e-8829-3b0bf6d35c93\") " pod="kube-system/konnectivity-agent-tj9nj" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931821 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5kxf\" (UniqueName: \"kubernetes.io/projected/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-kube-api-access-c5kxf\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931850 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s94z5\" (UniqueName: \"kubernetes.io/projected/0fc4604b-fd16-466e-8a28-e77878ba81cd-kube-api-access-s94z5\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931874 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-var-lib-cni-bin\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931900 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-host\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931925 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdwtp\" (UniqueName: \"kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp\") pod \"network-check-target-plvr8\" (UID: \"e754de95-041a-497a-bee1-8ca095b326f4\") " pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931949 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-cni-bin\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931973 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-run-k8s-cni-cncf-io\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.931998 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec3637df-91b4-4cbe-8d8b-7378b2c12a4f-host\") pod \"node-ca-hrgm6\" (UID: \"ec3637df-91b4-4cbe-8d8b-7378b2c12a4f\") " pod="openshift-image-registry/node-ca-hrgm6" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932021 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-tuned\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932046 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-systemd-units\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932070 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-slash\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.933169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932111 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-log-socket\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.934016 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932136 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.934016 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932163 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0fc4604b-fd16-466e-8a28-e77878ba81cd-ovnkube-script-lib\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.934016 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932210 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-multus-socket-dir-parent\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.934016 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932139 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a85534bb-a70d-4c02-a132-b56a2faf2c47-host\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.934016 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932256 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-cni-binary-copy\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:41.934016 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932291 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-run-netns\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.934016 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932324 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-run-systemd\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.934016 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932350 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-etc-openvswitch\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.934016 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932378 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-registration-dir\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:41.934016 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932407 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-etc-selinux\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:41.934016 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932432 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-dbus\") pod \"global-pull-secret-syncer-k8jtg\" (UID: \"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8\") " pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:41.934016 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932461 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btxvg\" (UniqueName: \"kubernetes.io/projected/5fbd07ff-dbb9-46f0-904d-d21419e31373-kube-api-access-btxvg\") pod \"network-metrics-daemon-f5cpd\" (UID: \"5fbd07ff-dbb9-46f0-904d-d21419e31373\") " pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:41.934016 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932484 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:41.934016 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932506 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0fc4604b-fd16-466e-8a28-e77878ba81cd-env-overrides\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:41.934016 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.932530 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-system-cni-dir\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:41.934892 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.934874 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a85534bb-a70d-4c02-a132-b56a2faf2c47-tmp\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.934950 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.934900 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a85534bb-a70d-4c02-a132-b56a2faf2c47-etc-tuned\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.942815 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:41.942793 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 13:00:41.942815 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:41.942817 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 13:00:41.942965 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:41.942829 2564 projected.go:196] Error preparing data for projected volume kube-api-access-sdwtp for pod openshift-network-diagnostics/network-check-target-plvr8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 13:00:41.942965 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:41.942895 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp podName:e754de95-041a-497a-bee1-8ca095b326f4 nodeName:}" failed. No retries permitted until 2026-09-07 13:00:42.442881421 +0000 UTC m=+3.038386017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sdwtp" (UniqueName: "kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp") pod "network-check-target-plvr8" (UID: "e754de95-041a-497a-bee1-8ca095b326f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 13:00:41.945744 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.945690 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npssx\" (UniqueName: \"kubernetes.io/projected/8ed5b1e0-eb99-433d-9df9-43019c49d31a-kube-api-access-npssx\") pod \"iptables-alerter-w27xg\" (UID: \"8ed5b1e0-eb99-433d-9df9-43019c49d31a\") " pod="openshift-network-operator/iptables-alerter-w27xg" Sep 07 13:00:41.946151 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.946114 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htj7j\" (UniqueName: \"kubernetes.io/projected/a85534bb-a70d-4c02-a132-b56a2faf2c47-kube-api-access-htj7j\") pod \"tuned-9rrnt\" (UID: \"a85534bb-a70d-4c02-a132-b56a2faf2c47\") " pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:41.946422 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.946403 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsrb9\" (UniqueName: \"kubernetes.io/projected/912823fa-6c24-493a-8640-89b921cdf2f2-kube-api-access-xsrb9\") pod \"node-resolver-fcd4k\" (UID: \"912823fa-6c24-493a-8640-89b921cdf2f2\") " pod="openshift-dns/node-resolver-fcd4k" Sep 07 13:00:41.946505 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:41.946478 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btxvg\" (UniqueName: \"kubernetes.io/projected/5fbd07ff-dbb9-46f0-904d-d21419e31373-kube-api-access-btxvg\") pod \"network-metrics-daemon-f5cpd\" (UID: \"5fbd07ff-dbb9-46f0-904d-d21419e31373\") " pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:42.033659 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033622 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-socket-dir\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:42.033659 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033659 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-var-lib-kubelet\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.033891 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033678 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-run-netns\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.033891 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033748 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-multus-cni-dir\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.033891 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033791 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-var-lib-kubelet\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.033891 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033797 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-cnibin\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.033891 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033841 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-hostroot\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.033891 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033855 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-socket-dir\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:42.033891 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033868 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-kubelet-config\") pod \"global-pull-secret-syncer-k8jtg\" (UID: \"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8\") " pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:42.033891 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033878 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-multus-cni-dir\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033897 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-run-openvswitch\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033903 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-hostroot\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033928 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-run-netns\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033933 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-run-ovn\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033943 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-run-openvswitch\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033943 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-kubelet-config\") pod \"global-pull-secret-syncer-k8jtg\" (UID: \"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8\") " pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033965 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-run-ovn\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033972 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-cni-netd\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.033983 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-cnibin\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034001 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0fc4604b-fd16-466e-8a28-e77878ba81cd-ovn-node-metrics-cert\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034028 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-run-multus-certs\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034061 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-cni-netd\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034085 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034119 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-kubelet\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034156 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0fc4604b-fd16-466e-8a28-e77878ba81cd-ovnkube-config\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034184 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034192 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-run-multus-certs\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.034261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034212 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034258 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-var-lib-openvswitch\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034287 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkxfm\" (UniqueName: \"kubernetes.io/projected/1b2a29b1-b954-4434-99d2-d8168c756c9f-kube-api-access-jkxfm\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034315 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ec3637df-91b4-4cbe-8d8b-7378b2c12a4f-serviceca\") pod \"node-ca-hrgm6\" (UID: \"ec3637df-91b4-4cbe-8d8b-7378b2c12a4f\") " pod="openshift-image-registry/node-ca-hrgm6" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034354 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034382 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-var-lib-cni-multus\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034410 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-multus-conf-dir\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034434 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmglh\" (UniqueName: \"kubernetes.io/projected/ec3637df-91b4-4cbe-8d8b-7378b2c12a4f-kube-api-access-vmglh\") pod \"node-ca-hrgm6\" (UID: \"ec3637df-91b4-4cbe-8d8b-7378b2c12a4f\") " pod="openshift-image-registry/node-ca-hrgm6" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034460 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-sys-fs\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034484 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/84352554-8f00-4dfe-8a59-c14dc986c2dd-cni-binary-copy\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034509 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-etc-kubernetes\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034539 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-cnibin\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034565 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-device-dir\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034590 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jplhk\" (UniqueName: \"kubernetes.io/projected/84352554-8f00-4dfe-8a59-c14dc986c2dd-kube-api-access-jplhk\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034622 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret\") pod \"global-pull-secret-syncer-k8jtg\" (UID: \"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8\") " pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034670 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-node-log\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034697 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/84352554-8f00-4dfe-8a59-c14dc986c2dd-multus-daemon-config\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.034988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034718 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034725 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3c09e344-284e-410e-8829-3b0bf6d35c93-agent-certs\") pod \"konnectivity-agent-tj9nj\" (UID: \"3c09e344-284e-410e-8829-3b0bf6d35c93\") " pod="kube-system/konnectivity-agent-tj9nj" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034753 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5kxf\" (UniqueName: \"kubernetes.io/projected/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-kube-api-access-c5kxf\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034778 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s94z5\" (UniqueName: \"kubernetes.io/projected/0fc4604b-fd16-466e-8a28-e77878ba81cd-kube-api-access-s94z5\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034802 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-sys-fs\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034805 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-var-lib-cni-bin\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034843 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-var-lib-cni-bin\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034842 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0fc4604b-fd16-466e-8a28-e77878ba81cd-ovnkube-config\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034857 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-cni-bin\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034886 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-run-k8s-cni-cncf-io\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034886 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-kubelet\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034915 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec3637df-91b4-4cbe-8d8b-7378b2c12a4f-host\") pod \"node-ca-hrgm6\" (UID: \"ec3637df-91b4-4cbe-8d8b-7378b2c12a4f\") " pod="openshift-image-registry/node-ca-hrgm6" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034943 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-systemd-units\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034971 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-slash\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.034975 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035018 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-slash\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035030 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-multus-conf-dir\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.035566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035061 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-var-lib-cni-multus\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035237 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035292 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-var-lib-openvswitch\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035315 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-cni-bin\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035364 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-host-run-k8s-cni-cncf-io\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035402 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/84352554-8f00-4dfe-8a59-c14dc986c2dd-cni-binary-copy\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035408 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec3637df-91b4-4cbe-8d8b-7378b2c12a4f-host\") pod \"node-ca-hrgm6\" (UID: \"ec3637df-91b4-4cbe-8d8b-7378b2c12a4f\") " pod="openshift-image-registry/node-ca-hrgm6" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035443 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-systemd-units\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035457 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035504 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-etc-kubernetes\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035548 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-cnibin\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035562 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-log-socket\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035596 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035604 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-device-dir\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035626 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0fc4604b-fd16-466e-8a28-e77878ba81cd-ovnkube-script-lib\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035655 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-multus-socket-dir-parent\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035682 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-cni-binary-copy\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:42.036187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035710 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-run-netns\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035733 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-run-systemd\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035757 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-etc-openvswitch\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035780 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-registration-dir\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035806 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-etc-selinux\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035831 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-dbus\") pod \"global-pull-secret-syncer-k8jtg\" (UID: \"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8\") " pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035833 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ec3637df-91b4-4cbe-8d8b-7378b2c12a4f-serviceca\") pod \"node-ca-hrgm6\" (UID: \"ec3637df-91b4-4cbe-8d8b-7378b2c12a4f\") " pod="openshift-image-registry/node-ca-hrgm6" Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:42.035871 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035887 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-log-socket\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035900 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-dbus\") pod \"global-pull-secret-syncer-k8jtg\" (UID: \"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8\") " pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035931 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:42.035940 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret podName:cefbef21-0cc5-4d7a-8dc3-e706bef2fba8 nodeName:}" failed. No retries permitted until 2026-09-07 13:00:42.535923101 +0000 UTC m=+3.131427727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret") pod "global-pull-secret-syncer-k8jtg" (UID: "cefbef21-0cc5-4d7a-8dc3-e706bef2fba8") : object "kube-system"/"original-pull-secret" not registered Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035962 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.035993 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0fc4604b-fd16-466e-8a28-e77878ba81cd-env-overrides\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.036019 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-system-cni-dir\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.036051 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-system-cni-dir\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.036079 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3c09e344-284e-410e-8829-3b0bf6d35c93-konnectivity-ca\") pod \"konnectivity-agent-tj9nj\" (UID: \"3c09e344-284e-410e-8829-3b0bf6d35c93\") " pod="kube-system/konnectivity-agent-tj9nj" Sep 07 13:00:42.037002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.036607 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0fc4604b-fd16-466e-8a28-e77878ba81cd-ovnkube-script-lib\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.037893 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.036632 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3c09e344-284e-410e-8829-3b0bf6d35c93-konnectivity-ca\") pod \"konnectivity-agent-tj9nj\" (UID: \"3c09e344-284e-410e-8829-3b0bf6d35c93\") " pod="kube-system/konnectivity-agent-tj9nj" Sep 07 13:00:42.037893 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.036686 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-multus-socket-dir-parent\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.037893 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.036873 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-etc-openvswitch\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.037893 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.036922 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-host-run-netns\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.037893 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.036963 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-run-systemd\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.037893 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.037018 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-registration-dir\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:42.037893 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.037070 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/84352554-8f00-4dfe-8a59-c14dc986c2dd-system-cni-dir\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.037893 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.037202 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1b2a29b1-b954-4434-99d2-d8168c756c9f-etc-selinux\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:42.037893 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.037223 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-cni-binary-copy\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:42.037893 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.037269 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0fc4604b-fd16-466e-8a28-e77878ba81cd-node-log\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.037893 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.037289 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-system-cni-dir\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:42.037893 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.037532 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:42.037893 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.037740 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0fc4604b-fd16-466e-8a28-e77878ba81cd-ovn-node-metrics-cert\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.037893 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.037869 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0fc4604b-fd16-466e-8a28-e77878ba81cd-env-overrides\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.038610 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.038219 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/84352554-8f00-4dfe-8a59-c14dc986c2dd-multus-daemon-config\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.039394 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.039372 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3c09e344-284e-410e-8829-3b0bf6d35c93-agent-certs\") pod \"konnectivity-agent-tj9nj\" (UID: \"3c09e344-284e-410e-8829-3b0bf6d35c93\") " pod="kube-system/konnectivity-agent-tj9nj" Sep 07 13:00:42.048432 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.048342 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkxfm\" (UniqueName: \"kubernetes.io/projected/1b2a29b1-b954-4434-99d2-d8168c756c9f-kube-api-access-jkxfm\") pod \"aws-ebs-csi-driver-node-bhzlh\" (UID: \"1b2a29b1-b954-4434-99d2-d8168c756c9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:42.048540 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.048451 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s94z5\" (UniqueName: \"kubernetes.io/projected/0fc4604b-fd16-466e-8a28-e77878ba81cd-kube-api-access-s94z5\") pod \"ovnkube-node-wxbzw\" (UID: \"0fc4604b-fd16-466e-8a28-e77878ba81cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.048695 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.048671 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmglh\" (UniqueName: \"kubernetes.io/projected/ec3637df-91b4-4cbe-8d8b-7378b2c12a4f-kube-api-access-vmglh\") pod \"node-ca-hrgm6\" (UID: \"ec3637df-91b4-4cbe-8d8b-7378b2c12a4f\") " pod="openshift-image-registry/node-ca-hrgm6" Sep 07 13:00:42.049281 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.049230 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5kxf\" (UniqueName: \"kubernetes.io/projected/568ca0aa-04bf-46b4-b3d1-38e935c0e5d8-kube-api-access-c5kxf\") pod \"multus-additional-cni-plugins-4p6gh\" (UID: \"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8\") " pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:42.049821 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.049803 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jplhk\" (UniqueName: \"kubernetes.io/projected/84352554-8f00-4dfe-8a59-c14dc986c2dd-kube-api-access-jplhk\") pod \"multus-8rdps\" (UID: \"84352554-8f00-4dfe-8a59-c14dc986c2dd\") " pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.113685 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.113651 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 13:00:42.118048 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.118019 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fcd4k" Sep 07 13:00:42.124635 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.124610 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w27xg" Sep 07 13:00:42.133988 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:42.133948 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod912823fa_6c24_493a_8640_89b921cdf2f2.slice/crio-01e66abdf940ad15f5cfc0e2964cc5af2a3f279ed6c0ae071498eb55c3ecb82e WatchSource:0}: Error finding container 01e66abdf940ad15f5cfc0e2964cc5af2a3f279ed6c0ae071498eb55c3ecb82e: Status 404 returned error can't find the container with id 01e66abdf940ad15f5cfc0e2964cc5af2a3f279ed6c0ae071498eb55c3ecb82e Sep 07 13:00:42.136735 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.136006 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" Sep 07 13:00:42.139489 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:42.139335 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ed5b1e0_eb99_433d_9df9_43019c49d31a.slice/crio-9ba608998b51a034fe93e4449ebfc40623e0221f25b35c2ab75f5093c61985fb WatchSource:0}: Error finding container 9ba608998b51a034fe93e4449ebfc40623e0221f25b35c2ab75f5093c61985fb: Status 404 returned error can't find the container with id 9ba608998b51a034fe93e4449ebfc40623e0221f25b35c2ab75f5093c61985fb Sep 07 13:00:42.140866 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.140708 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hrgm6" Sep 07 13:00:42.146340 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.146191 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:00:42.146596 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:42.146573 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda85534bb_a70d_4c02_a132_b56a2faf2c47.slice/crio-5a75bfedf8e7aacb392a6cacce1756324e2cb216d6852f0d0c5e9f1d2d80ef19 WatchSource:0}: Error finding container 5a75bfedf8e7aacb392a6cacce1756324e2cb216d6852f0d0c5e9f1d2d80ef19: Status 404 returned error can't find the container with id 5a75bfedf8e7aacb392a6cacce1756324e2cb216d6852f0d0c5e9f1d2d80ef19 Sep 07 13:00:42.152485 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.152460 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" Sep 07 13:00:42.152772 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:42.152743 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec3637df_91b4_4cbe_8d8b_7378b2c12a4f.slice/crio-9e3080d754295bfe3ffec220084c3d67ba9a4ab77fc206dedb212f6fdeae4f9f WatchSource:0}: Error finding container 9e3080d754295bfe3ffec220084c3d67ba9a4ab77fc206dedb212f6fdeae4f9f: Status 404 returned error can't find the container with id 9e3080d754295bfe3ffec220084c3d67ba9a4ab77fc206dedb212f6fdeae4f9f Sep 07 13:00:42.158156 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.158129 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8rdps" Sep 07 13:00:42.158613 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:42.158327 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fc4604b_fd16_466e_8a28_e77878ba81cd.slice/crio-bdd00bf8652a869836a41b820335df33bd9ea28964ca25c71ae0d1c9d56a64a0 WatchSource:0}: Error finding container bdd00bf8652a869836a41b820335df33bd9ea28964ca25c71ae0d1c9d56a64a0: Status 404 returned error can't find the container with id bdd00bf8652a869836a41b820335df33bd9ea28964ca25c71ae0d1c9d56a64a0 Sep 07 13:00:42.164156 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.164134 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tj9nj" Sep 07 13:00:42.170411 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.170210 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4p6gh" Sep 07 13:00:42.190028 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:42.189989 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c09e344_284e_410e_8829_3b0bf6d35c93.slice/crio-a31eb30709c1a706e1afbcc373f3d4d2064eb069e08fd7578cc2b19224ddbd74 WatchSource:0}: Error finding container a31eb30709c1a706e1afbcc373f3d4d2064eb069e08fd7578cc2b19224ddbd74: Status 404 returned error can't find the container with id a31eb30709c1a706e1afbcc373f3d4d2064eb069e08fd7578cc2b19224ddbd74 Sep 07 13:00:42.190633 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:00:42.190609 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod568ca0aa_04bf_46b4_b3d1_38e935c0e5d8.slice/crio-fa7df1f62e8b83abe9cd9034ebede8e9d9305d413de8c0a6303955ecc9116a6d WatchSource:0}: Error finding container fa7df1f62e8b83abe9cd9034ebede8e9d9305d413de8c0a6303955ecc9116a6d: Status 404 returned error can't find the container with id fa7df1f62e8b83abe9cd9034ebede8e9d9305d413de8c0a6303955ecc9116a6d Sep 07 13:00:42.438987 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.438894 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs\") pod \"network-metrics-daemon-f5cpd\" (UID: \"5fbd07ff-dbb9-46f0-904d-d21419e31373\") " pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:42.439169 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:42.439103 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 13:00:42.439232 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:42.439194 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs podName:5fbd07ff-dbb9-46f0-904d-d21419e31373 nodeName:}" failed. No retries permitted until 2026-09-07 13:00:43.439171141 +0000 UTC m=+4.034675756 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs") pod "network-metrics-daemon-f5cpd" (UID: "5fbd07ff-dbb9-46f0-904d-d21419e31373") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 13:00:42.540034 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.539988 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret\") pod \"global-pull-secret-syncer-k8jtg\" (UID: \"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8\") " pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:42.540034 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.540050 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdwtp\" (UniqueName: \"kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp\") pod \"network-check-target-plvr8\" (UID: \"e754de95-041a-497a-bee1-8ca095b326f4\") " pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:42.540300 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:42.540127 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 13:00:42.540300 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:42.540192 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 13:00:42.540300 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:42.540209 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 13:00:42.540300 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:42.540221 2564 projected.go:196] Error preparing data for projected volume kube-api-access-sdwtp for pod openshift-network-diagnostics/network-check-target-plvr8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 13:00:42.540300 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:42.540193 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret podName:cefbef21-0cc5-4d7a-8dc3-e706bef2fba8 nodeName:}" failed. No retries permitted until 2026-09-07 13:00:43.540174989 +0000 UTC m=+4.135679607 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret") pod "global-pull-secret-syncer-k8jtg" (UID: "cefbef21-0cc5-4d7a-8dc3-e706bef2fba8") : object "kube-system"/"original-pull-secret" not registered Sep 07 13:00:42.540539 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:42.540300 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp podName:e754de95-041a-497a-bee1-8ca095b326f4 nodeName:}" failed. No retries permitted until 2026-09-07 13:00:43.540267431 +0000 UTC m=+4.135772043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sdwtp" (UniqueName: "kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp") pod "network-check-target-plvr8" (UID: "e754de95-041a-497a-bee1-8ca095b326f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 13:00:42.869276 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.869147 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 12:55:42 +0000 UTC" deadline="2028-04-25 11:06:24.218388515 +0000 UTC" Sep 07 13:00:42.869276 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.869222 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14302h5m41.349170813s" Sep 07 13:00:42.916735 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.916699 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:42.916929 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:42.916843 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:00:42.926864 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.926813 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4p6gh" event={"ID":"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8","Type":"ContainerStarted","Data":"fa7df1f62e8b83abe9cd9034ebede8e9d9305d413de8c0a6303955ecc9116a6d"} Sep 07 13:00:42.928370 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.928330 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8rdps" event={"ID":"84352554-8f00-4dfe-8a59-c14dc986c2dd","Type":"ContainerStarted","Data":"813c9f90d2a0f849b3a2e33565804d766f4ca39a6aa5d57ebbbc2be04dccaace"} Sep 07 13:00:42.929742 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.929719 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" event={"ID":"1b2a29b1-b954-4434-99d2-d8168c756c9f","Type":"ContainerStarted","Data":"14c9d1b6f449ff63bd843e73b3a72a386105c0358a8654cd655001af4f723fde"} Sep 07 13:00:42.931222 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.931186 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" event={"ID":"0fc4604b-fd16-466e-8a28-e77878ba81cd","Type":"ContainerStarted","Data":"bdd00bf8652a869836a41b820335df33bd9ea28964ca25c71ae0d1c9d56a64a0"} Sep 07 13:00:42.932407 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.932382 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fcd4k" event={"ID":"912823fa-6c24-493a-8640-89b921cdf2f2","Type":"ContainerStarted","Data":"01e66abdf940ad15f5cfc0e2964cc5af2a3f279ed6c0ae071498eb55c3ecb82e"} Sep 07 13:00:42.933744 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.933717 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tj9nj" event={"ID":"3c09e344-284e-410e-8829-3b0bf6d35c93","Type":"ContainerStarted","Data":"a31eb30709c1a706e1afbcc373f3d4d2064eb069e08fd7578cc2b19224ddbd74"} Sep 07 13:00:42.935100 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.935076 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hrgm6" event={"ID":"ec3637df-91b4-4cbe-8d8b-7378b2c12a4f","Type":"ContainerStarted","Data":"9e3080d754295bfe3ffec220084c3d67ba9a4ab77fc206dedb212f6fdeae4f9f"} Sep 07 13:00:42.936486 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.936450 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" event={"ID":"a85534bb-a70d-4c02-a132-b56a2faf2c47","Type":"ContainerStarted","Data":"5a75bfedf8e7aacb392a6cacce1756324e2cb216d6852f0d0c5e9f1d2d80ef19"} Sep 07 13:00:42.937663 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:42.937637 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w27xg" event={"ID":"8ed5b1e0-eb99-433d-9df9-43019c49d31a","Type":"ContainerStarted","Data":"9ba608998b51a034fe93e4449ebfc40623e0221f25b35c2ab75f5093c61985fb"} Sep 07 13:00:43.448046 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:43.447821 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs\") pod \"network-metrics-daemon-f5cpd\" (UID: \"5fbd07ff-dbb9-46f0-904d-d21419e31373\") " pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:43.448046 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:43.448043 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 13:00:43.448348 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:43.448325 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs podName:5fbd07ff-dbb9-46f0-904d-d21419e31373 nodeName:}" failed. No retries permitted until 2026-09-07 13:00:45.448298748 +0000 UTC m=+6.043803360 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs") pod "network-metrics-daemon-f5cpd" (UID: "5fbd07ff-dbb9-46f0-904d-d21419e31373") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 13:00:43.548895 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:43.548829 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret\") pod \"global-pull-secret-syncer-k8jtg\" (UID: \"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8\") " pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:43.549087 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:43.548916 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdwtp\" (UniqueName: \"kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp\") pod \"network-check-target-plvr8\" (UID: \"e754de95-041a-497a-bee1-8ca095b326f4\") " pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:43.549213 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:43.549192 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 13:00:43.549263 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:43.549222 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 13:00:43.549305 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:43.549268 2564 projected.go:196] Error preparing data for projected volume kube-api-access-sdwtp for pod openshift-network-diagnostics/network-check-target-plvr8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 13:00:43.549388 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:43.549373 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp podName:e754de95-041a-497a-bee1-8ca095b326f4 nodeName:}" failed. No retries permitted until 2026-09-07 13:00:45.549345116 +0000 UTC m=+6.144849734 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sdwtp" (UniqueName: "kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp") pod "network-check-target-plvr8" (UID: "e754de95-041a-497a-bee1-8ca095b326f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 13:00:43.549936 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:43.549914 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 13:00:43.550027 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:43.549983 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret podName:cefbef21-0cc5-4d7a-8dc3-e706bef2fba8 nodeName:}" failed. No retries permitted until 2026-09-07 13:00:45.549964933 +0000 UTC m=+6.145469589 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret") pod "global-pull-secret-syncer-k8jtg" (UID: "cefbef21-0cc5-4d7a-8dc3-e706bef2fba8") : object "kube-system"/"original-pull-secret" not registered Sep 07 13:00:43.870339 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:43.870237 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 12:55:42 +0000 UTC" deadline="2028-05-20 15:20:42.492729536 +0000 UTC" Sep 07 13:00:43.870339 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:43.870286 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14906h19m58.622448959s" Sep 07 13:00:43.921730 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:43.921633 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:43.921730 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:43.921681 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:43.921993 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:43.921775 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:00:43.922283 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:43.922228 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:00:44.916989 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:44.916670 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:44.917503 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:44.917069 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:00:45.466536 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:45.466480 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs\") pod \"network-metrics-daemon-f5cpd\" (UID: \"5fbd07ff-dbb9-46f0-904d-d21419e31373\") " pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:45.466745 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:45.466700 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 13:00:45.466817 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:45.466761 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs podName:5fbd07ff-dbb9-46f0-904d-d21419e31373 nodeName:}" failed. No retries permitted until 2026-09-07 13:00:49.466741935 +0000 UTC m=+10.062246538 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs") pod "network-metrics-daemon-f5cpd" (UID: "5fbd07ff-dbb9-46f0-904d-d21419e31373") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 13:00:45.567288 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:45.567242 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret\") pod \"global-pull-secret-syncer-k8jtg\" (UID: \"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8\") " pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:45.567493 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:45.567307 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdwtp\" (UniqueName: \"kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp\") pod \"network-check-target-plvr8\" (UID: \"e754de95-041a-497a-bee1-8ca095b326f4\") " pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:45.567577 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:45.567553 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 13:00:45.567659 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:45.567637 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret podName:cefbef21-0cc5-4d7a-8dc3-e706bef2fba8 nodeName:}" failed. No retries permitted until 2026-09-07 13:00:49.567613744 +0000 UTC m=+10.163118363 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret") pod "global-pull-secret-syncer-k8jtg" (UID: "cefbef21-0cc5-4d7a-8dc3-e706bef2fba8") : object "kube-system"/"original-pull-secret" not registered Sep 07 13:00:45.568090 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:45.568067 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 13:00:45.568208 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:45.568096 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 13:00:45.568208 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:45.568113 2564 projected.go:196] Error preparing data for projected volume kube-api-access-sdwtp for pod openshift-network-diagnostics/network-check-target-plvr8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 13:00:45.568208 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:45.568176 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp podName:e754de95-041a-497a-bee1-8ca095b326f4 nodeName:}" failed. No retries permitted until 2026-09-07 13:00:49.568160501 +0000 UTC m=+10.163665112 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sdwtp" (UniqueName: "kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp") pod "network-check-target-plvr8" (UID: "e754de95-041a-497a-bee1-8ca095b326f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 13:00:45.919850 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:45.919765 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:45.920358 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:45.919919 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:00:45.920358 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:45.920334 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:45.920503 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:45.920442 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:00:46.916884 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:46.916353 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:46.916884 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:46.916491 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:00:46.966306 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:46.965528 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal" event={"ID":"d905437f79aa56bfe462c09feb96e548","Type":"ContainerStarted","Data":"4836282e6e3ecaed861148884f342ba3cffd7b760f919c92da5d85a0f6447845"} Sep 07 13:00:47.917233 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:47.916675 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:47.917233 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:47.916675 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:47.917233 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:47.916841 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:00:47.917233 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:47.916945 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:00:47.971912 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:47.971406 2564 generic.go:356] "Generic (PLEG): container finished" podID="d905437f79aa56bfe462c09feb96e548" containerID="4836282e6e3ecaed861148884f342ba3cffd7b760f919c92da5d85a0f6447845" exitCode=0 Sep 07 13:00:47.971912 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:47.971468 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal" event={"ID":"d905437f79aa56bfe462c09feb96e548","Type":"ContainerDied","Data":"4836282e6e3ecaed861148884f342ba3cffd7b760f919c92da5d85a0f6447845"} Sep 07 13:00:48.917289 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:48.917247 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:48.917498 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:48.917398 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:00:49.507754 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:49.507711 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs\") pod \"network-metrics-daemon-f5cpd\" (UID: \"5fbd07ff-dbb9-46f0-904d-d21419e31373\") " pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:49.508295 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:49.507910 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 13:00:49.508295 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:49.507995 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs podName:5fbd07ff-dbb9-46f0-904d-d21419e31373 nodeName:}" failed. No retries permitted until 2026-09-07 13:00:57.507972629 +0000 UTC m=+18.103477240 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs") pod "network-metrics-daemon-f5cpd" (UID: "5fbd07ff-dbb9-46f0-904d-d21419e31373") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 13:00:49.608763 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:49.608720 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret\") pod \"global-pull-secret-syncer-k8jtg\" (UID: \"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8\") " pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:49.608974 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:49.608785 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdwtp\" (UniqueName: \"kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp\") pod \"network-check-target-plvr8\" (UID: \"e754de95-041a-497a-bee1-8ca095b326f4\") " pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:49.608974 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:49.608925 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 13:00:49.608974 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:49.608936 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 13:00:49.608974 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:49.608956 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 13:00:49.608974 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:49.608968 2564 projected.go:196] Error preparing data for projected volume kube-api-access-sdwtp for pod openshift-network-diagnostics/network-check-target-plvr8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 13:00:49.609287 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:49.608995 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret podName:cefbef21-0cc5-4d7a-8dc3-e706bef2fba8 nodeName:}" failed. No retries permitted until 2026-09-07 13:00:57.608976168 +0000 UTC m=+18.204480790 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret") pod "global-pull-secret-syncer-k8jtg" (UID: "cefbef21-0cc5-4d7a-8dc3-e706bef2fba8") : object "kube-system"/"original-pull-secret" not registered Sep 07 13:00:49.609287 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:49.609016 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp podName:e754de95-041a-497a-bee1-8ca095b326f4 nodeName:}" failed. No retries permitted until 2026-09-07 13:00:57.60900412 +0000 UTC m=+18.204508717 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sdwtp" (UniqueName: "kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp") pod "network-check-target-plvr8" (UID: "e754de95-041a-497a-bee1-8ca095b326f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 13:00:49.920833 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:49.920744 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:49.921013 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:49.920886 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:00:49.921013 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:49.920941 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:49.921153 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:49.921095 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:00:50.917736 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:50.917245 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:50.917736 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:50.917379 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:00:51.921029 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:51.920296 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:51.921029 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:51.920427 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:00:51.921029 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:51.920890 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:51.921029 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:51.920986 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:00:52.916276 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:52.916236 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:52.916449 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:52.916367 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:00:53.917133 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:53.917100 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:53.917612 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:53.917100 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:53.917612 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:53.917243 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:00:53.917612 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:53.917286 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:00:54.916282 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:54.916244 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:54.916434 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:54.916370 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:00:55.919592 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:55.919564 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:55.919988 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:55.919666 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:00:55.919988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:55.919768 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:55.919988 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:55.919864 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:00:56.916644 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:56.916606 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:56.916842 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:56.916744 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:00:57.567295 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:57.567257 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs\") pod \"network-metrics-daemon-f5cpd\" (UID: \"5fbd07ff-dbb9-46f0-904d-d21419e31373\") " pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:57.567739 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:57.567453 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 13:00:57.567739 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:57.567537 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs podName:5fbd07ff-dbb9-46f0-904d-d21419e31373 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:13.567515625 +0000 UTC m=+34.163020233 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs") pod "network-metrics-daemon-f5cpd" (UID: "5fbd07ff-dbb9-46f0-904d-d21419e31373") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 13:00:57.667987 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:57.667944 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret\") pod \"global-pull-secret-syncer-k8jtg\" (UID: \"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8\") " pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:57.668204 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:57.668011 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdwtp\" (UniqueName: \"kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp\") pod \"network-check-target-plvr8\" (UID: \"e754de95-041a-497a-bee1-8ca095b326f4\") " pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:57.668204 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:57.668092 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 13:00:57.668204 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:57.668177 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret podName:cefbef21-0cc5-4d7a-8dc3-e706bef2fba8 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:13.668157123 +0000 UTC m=+34.263661731 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret") pod "global-pull-secret-syncer-k8jtg" (UID: "cefbef21-0cc5-4d7a-8dc3-e706bef2fba8") : object "kube-system"/"original-pull-secret" not registered Sep 07 13:00:57.668204 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:57.668177 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 13:00:57.668204 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:57.668207 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 13:00:57.668413 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:57.668221 2564 projected.go:196] Error preparing data for projected volume kube-api-access-sdwtp for pod openshift-network-diagnostics/network-check-target-plvr8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 13:00:57.668413 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:57.668257 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp podName:e754de95-041a-497a-bee1-8ca095b326f4 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:13.668246201 +0000 UTC m=+34.263750823 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sdwtp" (UniqueName: "kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp") pod "network-check-target-plvr8" (UID: "e754de95-041a-497a-bee1-8ca095b326f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 13:00:57.917137 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:57.917031 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:57.917137 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:57.917098 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:57.917363 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:57.917195 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:00:57.917363 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:57.917336 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:00:58.916733 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:58.916695 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:00:58.917229 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:58.916821 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:00:59.936159 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:59.936130 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:00:59.936159 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:00:59.936145 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:00:59.936629 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:59.936249 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:00:59.936629 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:00:59.936397 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:01:00.916916 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:00.916429 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:01:00.916916 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:00.916558 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:01:01.005106 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:01.005044 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-45.ec2.internal" event={"ID":"e4dcf97b5b5636889d8eabaae498c349","Type":"ContainerStarted","Data":"6909f437d26f2efc2c292a190af6e102006713b1e8a2af7bfcaf8caff93bf87b"} Sep 07 13:01:01.026817 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:01.026720 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-45.ec2.internal" podStartSLOduration=20.026698242 podStartE2EDuration="20.026698242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 13:00:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:01:01.024586345 +0000 UTC m=+21.620090961" watchObservedRunningTime="2026-09-07 13:01:01.026698242 +0000 UTC m=+21.622202859" Sep 07 13:01:01.920317 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:01.920124 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:01:01.920570 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:01.920124 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:01:01.920570 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:01.920407 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:01:01.920570 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:01.920470 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:01:02.026139 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.026103 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal" event={"ID":"d905437f79aa56bfe462c09feb96e548","Type":"ContainerStarted","Data":"64e2246afe6357342c4761eca318ef150554f22f3d15f4d12bfd7112a3e85409"} Sep 07 13:01:02.027600 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.027570 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tj9nj" event={"ID":"3c09e344-284e-410e-8829-3b0bf6d35c93","Type":"ContainerStarted","Data":"8a19602f74460ac3037504ca704d8a68d11567c7964c7c7557895f85a6affb51"} Sep 07 13:01:02.028992 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.028959 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hrgm6" event={"ID":"ec3637df-91b4-4cbe-8d8b-7378b2c12a4f","Type":"ContainerStarted","Data":"6195e50471f986b2791c365f56e3a7937ed1f0fc88f01a572856350a00228b53"} Sep 07 13:01:02.030508 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.030483 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" event={"ID":"a85534bb-a70d-4c02-a132-b56a2faf2c47","Type":"ContainerStarted","Data":"7def8389e7918e3b977b596cc5ffd7a628b3b6752aec7c37969c174c713974b3"} Sep 07 13:01:02.032359 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.032336 2564 generic.go:356] "Generic (PLEG): container finished" podID="568ca0aa-04bf-46b4-b3d1-38e935c0e5d8" containerID="4ba5df5079c3bd8148077a0aabf71076ec2a157a3a05de8151989db867e7604c" exitCode=0 Sep 07 13:01:02.032456 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.032415 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4p6gh" event={"ID":"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8","Type":"ContainerDied","Data":"4ba5df5079c3bd8148077a0aabf71076ec2a157a3a05de8151989db867e7604c"} Sep 07 13:01:02.033884 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.033830 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8rdps" event={"ID":"84352554-8f00-4dfe-8a59-c14dc986c2dd","Type":"ContainerStarted","Data":"d6bfc6305b98802d6dd631604d044db491bdc77137e7dcf2539418ab3eadd2ac"} Sep 07 13:01:02.035329 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.035306 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" event={"ID":"1b2a29b1-b954-4434-99d2-d8168c756c9f","Type":"ContainerStarted","Data":"a1b106da0f01368fe6198e723e89f8e3178eca8b78454fb003fe0aa68ec83498"} Sep 07 13:01:02.038273 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.038251 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" event={"ID":"0fc4604b-fd16-466e-8a28-e77878ba81cd","Type":"ContainerStarted","Data":"79c4b86ae7db1a6b0477f6517d53aaf2ae081dfc73471cae8897f05c404ff20d"} Sep 07 13:01:02.038380 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.038326 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" event={"ID":"0fc4604b-fd16-466e-8a28-e77878ba81cd","Type":"ContainerStarted","Data":"80b000672d2f08cc98edda11503d0e7b1a693a243f9a91f96579c875d259f7d7"} Sep 07 13:01:02.038425 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.038385 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" event={"ID":"0fc4604b-fd16-466e-8a28-e77878ba81cd","Type":"ContainerStarted","Data":"0da2042842a4f0469c4fe995770d45235f2f4eea2090878357c5ede01c008b19"} Sep 07 13:01:02.038459 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.038435 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" event={"ID":"0fc4604b-fd16-466e-8a28-e77878ba81cd","Type":"ContainerStarted","Data":"2ca8ebf78b338a85633b9c80e05dde3ea9608eaa1aed1853f2d18134f487584d"} Sep 07 13:01:02.038492 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.038482 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" event={"ID":"0fc4604b-fd16-466e-8a28-e77878ba81cd","Type":"ContainerStarted","Data":"ddd81079afdb0eeee0ea79eeffc7c3f95458700804cc9561df956dd3b304481e"} Sep 07 13:01:02.038545 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.038518 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" event={"ID":"0fc4604b-fd16-466e-8a28-e77878ba81cd","Type":"ContainerStarted","Data":"afe283b99a9e3493a4da80513502d936b295b8518f14189eba70c3ec8ba3599b"} Sep 07 13:01:02.039718 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.039692 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fcd4k" event={"ID":"912823fa-6c24-493a-8640-89b921cdf2f2","Type":"ContainerStarted","Data":"be5813132f4ffb91c56d8098129eee7b0da71b195ee7d66c7917cea599235aa4"} Sep 07 13:01:02.044124 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.044023 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-45.ec2.internal" podStartSLOduration=21.044010288 podStartE2EDuration="21.044010288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 13:00:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:01:02.043976479 +0000 UTC m=+22.639481095" watchObservedRunningTime="2026-09-07 13:01:02.044010288 +0000 UTC m=+22.639514904" Sep 07 13:01:02.073465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.073426 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fcd4k" podStartSLOduration=3.350432522 podStartE2EDuration="22.073401216s" totalImagesPullingTime="18.722968694s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 13:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:01:02.059157295 +0000 UTC m=+22.654661911" watchObservedRunningTime="2026-09-07 13:01:02.073401216 +0000 UTC m=+22.668905832" Sep 07 13:01:02.073540 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.073522 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hrgm6" podStartSLOduration=3.386616293 podStartE2EDuration="22.073514833s" totalImagesPullingTime="18.68689854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 13:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:01:02.073043135 +0000 UTC m=+22.668547752" watchObservedRunningTime="2026-09-07 13:01:02.073514833 +0000 UTC m=+22.669019448" Sep 07 13:01:02.090270 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.090225 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9rrnt" podStartSLOduration=3.377811792 podStartE2EDuration="22.090210878s" totalImagesPullingTime="18.712399086s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 13:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:01:02.08951329 +0000 UTC m=+22.685017916" watchObservedRunningTime="2026-09-07 13:01:02.090210878 +0000 UTC m=+22.685715494" Sep 07 13:01:02.112227 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.112162 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8rdps" podStartSLOduration=2.355709339 podStartE2EDuration="21.112140151s" totalImagesPullingTime="18.756430812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 13:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:01:02.111886929 +0000 UTC m=+22.707391545" watchObservedRunningTime="2026-09-07 13:01:02.112140151 +0000 UTC m=+22.707644770" Sep 07 13:01:02.144653 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.144607 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tj9nj" podStartSLOduration=2.759249317 podStartE2EDuration="21.144591133s" totalImagesPullingTime="18.385341816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:01:02.143966397 +0000 UTC m=+22.739471013" watchObservedRunningTime="2026-09-07 13:01:02.144591133 +0000 UTC m=+22.740095749" Sep 07 13:01:02.177099 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.177076 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 07 13:01:02.352301 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.352259 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tj9nj" Sep 07 13:01:02.352796 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.352780 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tj9nj" Sep 07 13:01:02.886535 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.886420 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-07T13:01:02.177095588Z","UUID":"7cb3003c-9d41-41a9-9c45-4e3a14c2a635","Handler":null,"Name":"","Endpoint":""} Sep 07 13:01:02.889653 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.889628 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 07 13:01:02.889653 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.889662 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 07 13:01:02.916944 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:02.916912 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:01:02.917155 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:02.917042 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:01:03.044085 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:03.044035 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w27xg" event={"ID":"8ed5b1e0-eb99-433d-9df9-43019c49d31a","Type":"ContainerStarted","Data":"1bd7dbb0618f19aea9978e98d2c8faab6b6a1492771c54429701bf96a2de0fe7"} Sep 07 13:01:03.046856 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:03.046828 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" event={"ID":"1b2a29b1-b954-4434-99d2-d8168c756c9f","Type":"ContainerStarted","Data":"9e9257540752584dec18d9b79662458ead3043896abac6f77ddcc3fbf939cd16"} Sep 07 13:01:03.047537 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:03.047519 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tj9nj" Sep 07 13:01:03.048092 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:03.048069 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tj9nj" Sep 07 13:01:03.060714 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:03.060676 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-w27xg" podStartSLOduration=4.361286706 podStartE2EDuration="23.060662557s" totalImagesPullingTime="18.699375851s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 13:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:01:03.060507201 +0000 UTC m=+23.656011817" watchObservedRunningTime="2026-09-07 13:01:03.060662557 +0000 UTC m=+23.656167197" Sep 07 13:01:03.919728 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:03.919659 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:01:03.919858 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:03.919751 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:01:03.919858 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:03.919776 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:01:03.919858 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:03.919828 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:01:04.053093 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:04.053040 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" event={"ID":"0fc4604b-fd16-466e-8a28-e77878ba81cd","Type":"ContainerStarted","Data":"c18801b5cb3e0d22ec220de6e01ab1eb08490b08cc8d99b2be66ba7feccac734"} Sep 07 13:01:04.916732 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:04.916690 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:01:04.916919 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:04.916836 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:01:05.919969 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:05.919937 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:01:05.920433 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:05.919937 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:01:05.920433 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:05.920100 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:01:05.920433 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:05.920178 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:01:06.917354 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:06.917123 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:01:06.917354 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:06.917243 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:01:07.063187 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:07.062855 2564 generic.go:356] "Generic (PLEG): container finished" podID="568ca0aa-04bf-46b4-b3d1-38e935c0e5d8" containerID="95a26c54e890c79633520c449d945ff106d490f5c74f924bdc06f56f2eb4ae1b" exitCode=0 Sep 07 13:01:07.063577 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:07.062931 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4p6gh" event={"ID":"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8","Type":"ContainerDied","Data":"95a26c54e890c79633520c449d945ff106d490f5c74f924bdc06f56f2eb4ae1b"} Sep 07 13:01:07.065702 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:07.065672 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" event={"ID":"1b2a29b1-b954-4434-99d2-d8168c756c9f","Type":"ContainerStarted","Data":"130710a0e192d2fd47df7f947caaf4b4f46fb87d5f460e26e7f36967f007584f"} Sep 07 13:01:07.069590 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:07.069568 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" event={"ID":"0fc4604b-fd16-466e-8a28-e77878ba81cd","Type":"ContainerStarted","Data":"c7e0eadde00f953f5f929139a612a326a23cda511b94b98b391e962827815d32"} Sep 07 13:01:07.069814 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:07.069800 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:01:07.069879 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:07.069858 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:01:07.084413 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:07.084393 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:01:07.114108 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:07.114046 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhzlh" podStartSLOduration=1.8981695360000002 podStartE2EDuration="26.114033877s" totalImagesPullingTime="24.215864341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 13:00:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:01:07.113806557 +0000 UTC m=+27.709311172" watchObservedRunningTime="2026-09-07 13:01:07.114033877 +0000 UTC m=+27.709538531" Sep 07 13:01:07.146145 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:07.145074 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" podStartSLOduration=7.382802161 podStartE2EDuration="26.145029907s" totalImagesPullingTime="18.762227746s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 13:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:01:07.144378747 +0000 UTC m=+27.739883361" watchObservedRunningTime="2026-09-07 13:01:07.145029907 +0000 UTC m=+27.740534523" Sep 07 13:01:07.917259 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:07.917231 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:01:07.917418 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:07.917242 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:01:07.917418 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:07.917326 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:01:07.917418 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:07.917407 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:01:08.072995 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:08.072917 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:01:08.090161 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:08.090128 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:01:08.916937 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:08.916905 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:01:08.917107 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:08.917003 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:01:09.019682 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:09.019653 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k8jtg"] Sep 07 13:01:09.019804 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:09.019743 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:01:09.019858 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:09.019832 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:01:09.022821 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:09.022750 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-plvr8"] Sep 07 13:01:09.023404 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:09.023385 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f5cpd"] Sep 07 13:01:09.023508 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:09.023475 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:01:09.023604 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:09.023583 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:01:09.076807 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:09.076775 2564 generic.go:356] "Generic (PLEG): container finished" podID="568ca0aa-04bf-46b4-b3d1-38e935c0e5d8" containerID="a23f7ce913c474034804531e6fa582282098e11c1b3a40e49a77562dd5fe0105" exitCode=0 Sep 07 13:01:09.077533 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:09.076868 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4p6gh" event={"ID":"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8","Type":"ContainerDied","Data":"a23f7ce913c474034804531e6fa582282098e11c1b3a40e49a77562dd5fe0105"} Sep 07 13:01:09.077533 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:09.076918 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:01:09.077533 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:09.077163 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:01:10.081887 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:10.081855 2564 generic.go:356] "Generic (PLEG): container finished" podID="568ca0aa-04bf-46b4-b3d1-38e935c0e5d8" containerID="fcc9f926b2f609b7d58dc077e9f1faaeffa0d66c262d0e7445240b893138cc5a" exitCode=0 Sep 07 13:01:10.082342 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:10.081945 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4p6gh" event={"ID":"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8","Type":"ContainerDied","Data":"fcc9f926b2f609b7d58dc077e9f1faaeffa0d66c262d0e7445240b893138cc5a"} Sep 07 13:01:10.916984 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:10.916718 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:01:10.916984 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:10.916718 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:01:10.917219 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:10.917006 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:01:10.917219 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:10.916759 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:01:10.917219 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:10.917047 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:01:10.917219 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:10.917154 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:01:12.916489 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:12.916414 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:01:12.916954 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:12.916414 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:01:12.916954 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:12.916533 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-plvr8" podUID="e754de95-041a-497a-bee1-8ca095b326f4" Sep 07 13:01:12.916954 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:12.916414 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:01:12.916954 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:12.916602 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8jtg" podUID="cefbef21-0cc5-4d7a-8dc3-e706bef2fba8" Sep 07 13:01:12.916954 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:12.916719 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5cpd" podUID="5fbd07ff-dbb9-46f0-904d-d21419e31373" Sep 07 13:01:13.580070 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:13.580017 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs\") pod \"network-metrics-daemon-f5cpd\" (UID: \"5fbd07ff-dbb9-46f0-904d-d21419e31373\") " pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:01:13.580250 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:13.580192 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 13:01:13.580309 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:13.580272 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs podName:5fbd07ff-dbb9-46f0-904d-d21419e31373 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:45.580253129 +0000 UTC m=+66.175757727 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs") pod "network-metrics-daemon-f5cpd" (UID: "5fbd07ff-dbb9-46f0-904d-d21419e31373") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 13:01:13.681209 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:13.681153 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret\") pod \"global-pull-secret-syncer-k8jtg\" (UID: \"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8\") " pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:01:13.681209 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:13.681212 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdwtp\" (UniqueName: \"kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp\") pod \"network-check-target-plvr8\" (UID: \"e754de95-041a-497a-bee1-8ca095b326f4\") " pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:01:13.681466 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:13.681338 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 13:01:13.681466 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:13.681357 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 13:01:13.681466 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:13.681380 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 13:01:13.681466 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:13.681394 2564 projected.go:196] Error preparing data for projected volume kube-api-access-sdwtp for pod openshift-network-diagnostics/network-check-target-plvr8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 13:01:13.681466 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:13.681421 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret podName:cefbef21-0cc5-4d7a-8dc3-e706bef2fba8 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:45.681401369 +0000 UTC m=+66.276905966 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret") pod "global-pull-secret-syncer-k8jtg" (UID: "cefbef21-0cc5-4d7a-8dc3-e706bef2fba8") : object "kube-system"/"original-pull-secret" not registered Sep 07 13:01:13.681466 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:13.681446 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp podName:e754de95-041a-497a-bee1-8ca095b326f4 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:45.681430179 +0000 UTC m=+66.276934775 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sdwtp" (UniqueName: "kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp") pod "network-check-target-plvr8" (UID: "e754de95-041a-497a-bee1-8ca095b326f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 13:01:14.705230 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.705161 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-45.ec2.internal" event="NodeReady" Sep 07 13:01:14.705645 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.705305 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 07 13:01:14.747953 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.747921 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-864459d56-bbg5b"] Sep 07 13:01:14.777324 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.777291 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-74wz2"] Sep 07 13:01:14.777492 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.777406 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.780087 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.779989 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fsnx6\"" Sep 07 13:01:14.780202 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.780185 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 07 13:01:14.780445 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.780427 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 07 13:01:14.781068 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.781033 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 07 13:01:14.786849 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.786830 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 07 13:01:14.793186 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.793165 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6gpjx"] Sep 07 13:01:14.793306 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.793286 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:14.796869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.795808 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-77fxd\"" Sep 07 13:01:14.796869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.796076 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 07 13:01:14.796869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.796309 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 07 13:01:14.813345 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.813323 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-864459d56-bbg5b"] Sep 07 13:01:14.813345 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.813354 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-74wz2"] Sep 07 13:01:14.813514 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.813378 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6gpjx"] Sep 07 13:01:14.813514 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.813456 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6gpjx" Sep 07 13:01:14.815835 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.815813 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 07 13:01:14.815938 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.815836 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 07 13:01:14.815938 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.815889 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-p7csz\"" Sep 07 13:01:14.815938 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.815839 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 07 13:01:14.889950 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.889903 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtfjc\" (UniqueName: \"kubernetes.io/projected/66d7fa5d-e887-46a9-b527-a7a8398e4786-kube-api-access-xtfjc\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:14.889950 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.889940 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b87e5f6-90fc-4891-aa11-feb499ce5511-installation-pull-secrets\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.890193 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.889967 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66d7fa5d-e887-46a9-b527-a7a8398e4786-tmp-dir\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:14.890193 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.890015 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87r5w\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-kube-api-access-87r5w\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.890193 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.890080 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xm2j\" (UniqueName: \"kubernetes.io/projected/3f13b2c1-d6fb-4722-b60e-4b940d617447-kube-api-access-2xm2j\") pod \"ingress-canary-6gpjx\" (UID: \"3f13b2c1-d6fb-4722-b60e-4b940d617447\") " pod="openshift-ingress-canary/ingress-canary-6gpjx" Sep 07 13:01:14.890193 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.890173 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-certificates\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.890370 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.890213 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.890370 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.890236 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/66d7fa5d-e887-46a9-b527-a7a8398e4786-config-volume\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:14.890370 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.890265 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-bound-sa-token\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.890370 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.890304 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b87e5f6-90fc-4891-aa11-feb499ce5511-ca-trust-extracted\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.890370 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.890344 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:14.890370 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.890370 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f13b2c1-d6fb-4722-b60e-4b940d617447-cert\") pod \"ingress-canary-6gpjx\" (UID: \"3f13b2c1-d6fb-4722-b60e-4b940d617447\") " pod="openshift-ingress-canary/ingress-canary-6gpjx" Sep 07 13:01:14.890606 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.890429 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b87e5f6-90fc-4891-aa11-feb499ce5511-image-registry-private-configuration\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.890606 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.890454 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b87e5f6-90fc-4891-aa11-feb499ce5511-trusted-ca\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.917295 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.917257 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:01:14.917295 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.917273 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:01:14.917525 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.917273 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:01:14.920434 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.920216 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 07 13:01:14.920434 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.920248 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 07 13:01:14.920434 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.920302 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 07 13:01:14.920434 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.920252 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 07 13:01:14.920720 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.920449 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x9nqz\"" Sep 07 13:01:14.920720 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.920601 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sgcc8\"" Sep 07 13:01:14.991621 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.991590 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/66d7fa5d-e887-46a9-b527-a7a8398e4786-config-volume\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:14.991621 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.991627 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-bound-sa-token\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.991862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.991646 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b87e5f6-90fc-4891-aa11-feb499ce5511-ca-trust-extracted\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.991862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.991668 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:14.991862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.991691 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f13b2c1-d6fb-4722-b60e-4b940d617447-cert\") pod \"ingress-canary-6gpjx\" (UID: \"3f13b2c1-d6fb-4722-b60e-4b940d617447\") " pod="openshift-ingress-canary/ingress-canary-6gpjx" Sep 07 13:01:14.991862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.991730 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b87e5f6-90fc-4891-aa11-feb499ce5511-image-registry-private-configuration\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.991862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.991747 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b87e5f6-90fc-4891-aa11-feb499ce5511-trusted-ca\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.991862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.991772 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtfjc\" (UniqueName: \"kubernetes.io/projected/66d7fa5d-e887-46a9-b527-a7a8398e4786-kube-api-access-xtfjc\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:14.991862 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:14.991784 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 13:01:14.991862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.991799 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b87e5f6-90fc-4891-aa11-feb499ce5511-installation-pull-secrets\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.991862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.991827 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66d7fa5d-e887-46a9-b527-a7a8398e4786-tmp-dir\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:14.991862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.991851 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87r5w\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-kube-api-access-87r5w\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.991862 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:14.991867 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls podName:66d7fa5d-e887-46a9-b527-a7a8398e4786 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:15.491845878 +0000 UTC m=+36.087350491 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls") pod "dns-default-74wz2" (UID: "66d7fa5d-e887-46a9-b527-a7a8398e4786") : secret "dns-default-metrics-tls" not found Sep 07 13:01:14.992355 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.991907 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xm2j\" (UniqueName: \"kubernetes.io/projected/3f13b2c1-d6fb-4722-b60e-4b940d617447-kube-api-access-2xm2j\") pod \"ingress-canary-6gpjx\" (UID: \"3f13b2c1-d6fb-4722-b60e-4b940d617447\") " pod="openshift-ingress-canary/ingress-canary-6gpjx" Sep 07 13:01:14.992355 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.991962 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-certificates\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.992355 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.992011 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.992355 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.992077 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b87e5f6-90fc-4891-aa11-feb499ce5511-ca-trust-extracted\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.992355 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:14.992186 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 13:01:14.992355 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:14.992201 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864459d56-bbg5b: secret "image-registry-tls" not found Sep 07 13:01:14.992355 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:14.992246 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls podName:4b87e5f6-90fc-4891-aa11-feb499ce5511 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:15.492229621 +0000 UTC m=+36.087734235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls") pod "image-registry-864459d56-bbg5b" (UID: "4b87e5f6-90fc-4891-aa11-feb499ce5511") : secret "image-registry-tls" not found Sep 07 13:01:14.992355 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.992283 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/66d7fa5d-e887-46a9-b527-a7a8398e4786-config-volume\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:14.992761 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.992574 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-certificates\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.992761 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.992566 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66d7fa5d-e887-46a9-b527-a7a8398e4786-tmp-dir\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:14.992852 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.992832 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b87e5f6-90fc-4891-aa11-feb499ce5511-trusted-ca\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.995783 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.995761 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b87e5f6-90fc-4891-aa11-feb499ce5511-installation-pull-secrets\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:14.995892 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.995770 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f13b2c1-d6fb-4722-b60e-4b940d617447-cert\") pod \"ingress-canary-6gpjx\" (UID: \"3f13b2c1-d6fb-4722-b60e-4b940d617447\") " pod="openshift-ingress-canary/ingress-canary-6gpjx" Sep 07 13:01:14.995892 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:14.995837 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b87e5f6-90fc-4891-aa11-feb499ce5511-image-registry-private-configuration\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:15.003331 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:15.003305 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-bound-sa-token\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:15.003449 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:15.003338 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xm2j\" (UniqueName: \"kubernetes.io/projected/3f13b2c1-d6fb-4722-b60e-4b940d617447-kube-api-access-2xm2j\") pod \"ingress-canary-6gpjx\" (UID: \"3f13b2c1-d6fb-4722-b60e-4b940d617447\") " pod="openshift-ingress-canary/ingress-canary-6gpjx" Sep 07 13:01:15.003589 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:15.003567 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtfjc\" (UniqueName: \"kubernetes.io/projected/66d7fa5d-e887-46a9-b527-a7a8398e4786-kube-api-access-xtfjc\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:15.003690 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:15.003621 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87r5w\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-kube-api-access-87r5w\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:15.124002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:15.123963 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6gpjx" Sep 07 13:01:15.429847 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:15.429819 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6gpjx"] Sep 07 13:01:15.433623 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:01:15.433592 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f13b2c1_d6fb_4722_b60e_4b940d617447.slice/crio-fe611dbc34d7157df50cdeeaab99d6aa180c1e7037a9e7c4ce7982dd449be8ca WatchSource:0}: Error finding container fe611dbc34d7157df50cdeeaab99d6aa180c1e7037a9e7c4ce7982dd449be8ca: Status 404 returned error can't find the container with id fe611dbc34d7157df50cdeeaab99d6aa180c1e7037a9e7c4ce7982dd449be8ca Sep 07 13:01:15.495814 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:15.495789 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:15.495944 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:15.495864 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:15.495944 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:15.495925 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 13:01:15.496013 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:15.495949 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 13:01:15.496013 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:15.495959 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864459d56-bbg5b: secret "image-registry-tls" not found Sep 07 13:01:15.496013 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:15.495986 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls podName:66d7fa5d-e887-46a9-b527-a7a8398e4786 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:16.495971603 +0000 UTC m=+37.091476196 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls") pod "dns-default-74wz2" (UID: "66d7fa5d-e887-46a9-b527-a7a8398e4786") : secret "dns-default-metrics-tls" not found Sep 07 13:01:15.496013 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:15.495999 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls podName:4b87e5f6-90fc-4891-aa11-feb499ce5511 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:16.495993644 +0000 UTC m=+37.091498239 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls") pod "image-registry-864459d56-bbg5b" (UID: "4b87e5f6-90fc-4891-aa11-feb499ce5511") : secret "image-registry-tls" not found Sep 07 13:01:16.097368 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:16.097337 2564 generic.go:356] "Generic (PLEG): container finished" podID="568ca0aa-04bf-46b4-b3d1-38e935c0e5d8" containerID="b8d7730a0f866d479b943e8f9dd29d49aed68df25334b12a89fd10a3d50f69e8" exitCode=0 Sep 07 13:01:16.097816 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:16.097422 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4p6gh" event={"ID":"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8","Type":"ContainerDied","Data":"b8d7730a0f866d479b943e8f9dd29d49aed68df25334b12a89fd10a3d50f69e8"} Sep 07 13:01:16.098507 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:16.098483 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6gpjx" event={"ID":"3f13b2c1-d6fb-4722-b60e-4b940d617447","Type":"ContainerStarted","Data":"fe611dbc34d7157df50cdeeaab99d6aa180c1e7037a9e7c4ce7982dd449be8ca"} Sep 07 13:01:16.502604 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:16.502560 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:16.502762 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:16.502689 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:16.502821 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:16.502760 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 13:01:16.502854 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:16.502847 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls podName:66d7fa5d-e887-46a9-b527-a7a8398e4786 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:18.502824939 +0000 UTC m=+39.098329547 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls") pod "dns-default-74wz2" (UID: "66d7fa5d-e887-46a9-b527-a7a8398e4786") : secret "dns-default-metrics-tls" not found Sep 07 13:01:16.502895 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:16.502851 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 13:01:16.502895 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:16.502869 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864459d56-bbg5b: secret "image-registry-tls" not found Sep 07 13:01:16.502960 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:16.502940 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls podName:4b87e5f6-90fc-4891-aa11-feb499ce5511 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:18.50292247 +0000 UTC m=+39.098427086 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls") pod "image-registry-864459d56-bbg5b" (UID: "4b87e5f6-90fc-4891-aa11-feb499ce5511") : secret "image-registry-tls" not found Sep 07 13:01:17.103776 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:17.103743 2564 generic.go:356] "Generic (PLEG): container finished" podID="568ca0aa-04bf-46b4-b3d1-38e935c0e5d8" containerID="87d01b8beef02056a47707af163003cb806975d19d66c1c6aceaf513dfcb7422" exitCode=0 Sep 07 13:01:17.104243 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:17.103791 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4p6gh" event={"ID":"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8","Type":"ContainerDied","Data":"87d01b8beef02056a47707af163003cb806975d19d66c1c6aceaf513dfcb7422"} Sep 07 13:01:18.110009 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:18.109919 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4p6gh" event={"ID":"568ca0aa-04bf-46b4-b3d1-38e935c0e5d8","Type":"ContainerStarted","Data":"33310f184337401a084748e903d2ba293c186f650337669d1e76655541e70e21"} Sep 07 13:01:18.135512 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:18.135398 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4p6gh" podStartSLOduration=6.165282129 podStartE2EDuration="37.135379456s" totalImagesPullingTime="30.970097327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 13:00:41 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:01:18.134032745 +0000 UTC m=+38.729537361" watchObservedRunningTime="2026-09-07 13:01:18.135379456 +0000 UTC m=+38.730884072" Sep 07 13:01:18.520264 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:18.520232 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:18.520397 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:18.520280 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:18.520397 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:18.520385 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 13:01:18.520465 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:18.520409 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864459d56-bbg5b: secret "image-registry-tls" not found Sep 07 13:01:18.520465 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:18.520459 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls podName:4b87e5f6-90fc-4891-aa11-feb499ce5511 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:22.520445 +0000 UTC m=+43.115949611 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls") pod "image-registry-864459d56-bbg5b" (UID: "4b87e5f6-90fc-4891-aa11-feb499ce5511") : secret "image-registry-tls" not found Sep 07 13:01:18.520538 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:18.520390 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 13:01:18.520538 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:18.520513 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls podName:66d7fa5d-e887-46a9-b527-a7a8398e4786 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:22.520502103 +0000 UTC m=+43.116006735 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls") pod "dns-default-74wz2" (UID: "66d7fa5d-e887-46a9-b527-a7a8398e4786") : secret "dns-default-metrics-tls" not found Sep 07 13:01:19.113690 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:19.113651 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6gpjx" event={"ID":"3f13b2c1-d6fb-4722-b60e-4b940d617447","Type":"ContainerStarted","Data":"aa2f88d752ef288d0b0ddc85e1016189f78bf19a5a347582839280fbe2fce51e"} Sep 07 13:01:19.131607 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:19.131548 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6gpjx" podStartSLOduration=2.100941385 podStartE2EDuration="5.131530074s" totalImagesPullingTime="3.030588689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:01:19.130786898 +0000 UTC m=+39.726291514" watchObservedRunningTime="2026-09-07 13:01:19.131530074 +0000 UTC m=+39.727034692" Sep 07 13:01:22.545813 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:22.545768 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:22.546235 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:22.545855 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:22.546235 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:22.545934 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 13:01:22.546235 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:22.546003 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls podName:66d7fa5d-e887-46a9-b527-a7a8398e4786 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:30.545987944 +0000 UTC m=+51.141492537 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls") pod "dns-default-74wz2" (UID: "66d7fa5d-e887-46a9-b527-a7a8398e4786") : secret "dns-default-metrics-tls" not found Sep 07 13:01:22.546235 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:22.545946 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 13:01:22.546235 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:22.546043 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864459d56-bbg5b: secret "image-registry-tls" not found Sep 07 13:01:22.546235 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:22.546106 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls podName:4b87e5f6-90fc-4891-aa11-feb499ce5511 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:30.546095363 +0000 UTC m=+51.141599957 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls") pod "image-registry-864459d56-bbg5b" (UID: "4b87e5f6-90fc-4891-aa11-feb499ce5511") : secret "image-registry-tls" not found Sep 07 13:01:30.600656 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:30.600611 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:30.601139 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:30.600711 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:30.601139 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:30.600761 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 13:01:30.601139 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:30.600803 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 13:01:30.601139 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:30.600816 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864459d56-bbg5b: secret "image-registry-tls" not found Sep 07 13:01:30.601139 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:30.600838 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls podName:66d7fa5d-e887-46a9-b527-a7a8398e4786 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:46.600820948 +0000 UTC m=+67.196325547 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls") pod "dns-default-74wz2" (UID: "66d7fa5d-e887-46a9-b527-a7a8398e4786") : secret "dns-default-metrics-tls" not found Sep 07 13:01:30.601139 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:30.600866 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls podName:4b87e5f6-90fc-4891-aa11-feb499ce5511 nodeName:}" failed. No retries permitted until 2026-09-07 13:01:46.600854446 +0000 UTC m=+67.196359039 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls") pod "image-registry-864459d56-bbg5b" (UID: "4b87e5f6-90fc-4891-aa11-feb499ce5511") : secret "image-registry-tls" not found Sep 07 13:01:40.092265 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:40.092237 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wxbzw" Sep 07 13:01:45.610764 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:45.610727 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs\") pod \"network-metrics-daemon-f5cpd\" (UID: \"5fbd07ff-dbb9-46f0-904d-d21419e31373\") " pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:01:45.613412 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:45.613385 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 07 13:01:45.621754 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:45.621729 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 07 13:01:45.621856 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:45.621792 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs podName:5fbd07ff-dbb9-46f0-904d-d21419e31373 nodeName:}" failed. No retries permitted until 2026-09-07 13:02:49.621776998 +0000 UTC m=+130.217281605 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs") pod "network-metrics-daemon-f5cpd" (UID: "5fbd07ff-dbb9-46f0-904d-d21419e31373") : secret "metrics-daemon-secret" not found Sep 07 13:01:45.711526 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:45.711496 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret\") pod \"global-pull-secret-syncer-k8jtg\" (UID: \"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8\") " pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:01:45.711662 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:45.711539 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdwtp\" (UniqueName: \"kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp\") pod \"network-check-target-plvr8\" (UID: \"e754de95-041a-497a-bee1-8ca095b326f4\") " pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:01:45.714240 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:45.714212 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 07 13:01:45.714348 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:45.714327 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 07 13:01:45.725015 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:45.724994 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 07 13:01:45.725431 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:45.725409 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cefbef21-0cc5-4d7a-8dc3-e706bef2fba8-original-pull-secret\") pod \"global-pull-secret-syncer-k8jtg\" (UID: \"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8\") " pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:01:45.735482 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:45.735453 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdwtp\" (UniqueName: \"kubernetes.io/projected/e754de95-041a-497a-bee1-8ca095b326f4-kube-api-access-sdwtp\") pod \"network-check-target-plvr8\" (UID: \"e754de95-041a-497a-bee1-8ca095b326f4\") " pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:01:45.831862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:45.831836 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x9nqz\"" Sep 07 13:01:45.839226 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:45.839203 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:01:45.841498 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:45.841479 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8jtg" Sep 07 13:01:46.144782 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:46.144697 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-plvr8"] Sep 07 13:01:46.145756 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:46.145665 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k8jtg"] Sep 07 13:01:46.148317 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:01:46.148291 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcefbef21_0cc5_4d7a_8dc3_e706bef2fba8.slice/crio-cdd6ed73935f3055b286a4780c8c9caf2fa0f1afb0947a57cc2678b11cbb118b WatchSource:0}: Error finding container cdd6ed73935f3055b286a4780c8c9caf2fa0f1afb0947a57cc2678b11cbb118b: Status 404 returned error can't find the container with id cdd6ed73935f3055b286a4780c8c9caf2fa0f1afb0947a57cc2678b11cbb118b Sep 07 13:01:46.149202 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:01:46.149174 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode754de95_041a_497a_bee1_8ca095b326f4.slice/crio-5de3c120a69019f0476c562ac7a7c5bb391fa4e6d2dc01a9b6be5fe345d9baba WatchSource:0}: Error finding container 5de3c120a69019f0476c562ac7a7c5bb391fa4e6d2dc01a9b6be5fe345d9baba: Status 404 returned error can't find the container with id 5de3c120a69019f0476c562ac7a7c5bb391fa4e6d2dc01a9b6be5fe345d9baba Sep 07 13:01:46.177635 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:46.177612 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k8jtg" event={"ID":"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8","Type":"ContainerStarted","Data":"cdd6ed73935f3055b286a4780c8c9caf2fa0f1afb0947a57cc2678b11cbb118b"} Sep 07 13:01:46.178685 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:46.178662 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-plvr8" event={"ID":"e754de95-041a-497a-bee1-8ca095b326f4","Type":"ContainerStarted","Data":"5de3c120a69019f0476c562ac7a7c5bb391fa4e6d2dc01a9b6be5fe345d9baba"} Sep 07 13:01:46.618047 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:46.618008 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:01:46.618517 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:46.618089 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:01:46.618517 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:46.618185 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 13:01:46.618517 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:46.618209 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-864459d56-bbg5b: secret "image-registry-tls" not found Sep 07 13:01:46.618517 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:46.618226 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 13:01:46.618517 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:46.618272 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls podName:4b87e5f6-90fc-4891-aa11-feb499ce5511 nodeName:}" failed. No retries permitted until 2026-09-07 13:02:18.618253935 +0000 UTC m=+99.213758548 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls") pod "image-registry-864459d56-bbg5b" (UID: "4b87e5f6-90fc-4891-aa11-feb499ce5511") : secret "image-registry-tls" not found Sep 07 13:01:46.618517 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:01:46.618287 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls podName:66d7fa5d-e887-46a9-b527-a7a8398e4786 nodeName:}" failed. No retries permitted until 2026-09-07 13:02:18.618279372 +0000 UTC m=+99.213783966 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls") pod "dns-default-74wz2" (UID: "66d7fa5d-e887-46a9-b527-a7a8398e4786") : secret "dns-default-metrics-tls" not found Sep 07 13:01:51.193907 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:51.193873 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-plvr8" event={"ID":"e754de95-041a-497a-bee1-8ca095b326f4","Type":"ContainerStarted","Data":"a3aec23fc7ef1dc9a20c03ad4e0930b7398ad434d575d379610d686bbe8e708a"} Sep 07 13:01:51.194343 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:51.193955 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:01:51.195500 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:51.195468 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k8jtg" event={"ID":"cefbef21-0cc5-4d7a-8dc3-e706bef2fba8","Type":"ContainerStarted","Data":"bd7cabcc6dadbcda4b4e8720782e8445522fe8af08d0d93c2558fe147bcd335a"} Sep 07 13:01:51.212658 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:51.212605 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-plvr8" podStartSLOduration=66.810322963 podStartE2EDuration="1m11.21258785s" totalImagesPullingTime="4.402264887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:01:51.211838476 +0000 UTC m=+71.807343106" watchObservedRunningTime="2026-09-07 13:01:51.21258785 +0000 UTC m=+71.808092466" Sep 07 13:01:51.227258 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:01:51.227205 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k8jtg" podStartSLOduration=65.819840437 podStartE2EDuration="1m10.227186001s" totalImagesPullingTime="4.407345564s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 13:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:01:51.226603159 +0000 UTC m=+71.822107774" watchObservedRunningTime="2026-09-07 13:01:51.227186001 +0000 UTC m=+71.822690618" Sep 07 13:02:01.636968 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.636932 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fwckf"] Sep 07 13:02:01.640964 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.640947 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w"] Sep 07 13:02:01.641108 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.641089 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fwckf" Sep 07 13:02:01.643485 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.643459 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 07 13:02:01.643616 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.643599 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-swqb6\"" Sep 07 13:02:01.643752 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.643734 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w" Sep 07 13:02:01.644626 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.644604 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 07 13:02:01.647323 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.647301 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 07 13:02:01.647493 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.647328 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 07 13:02:01.647493 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.647398 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 07 13:02:01.647992 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.647389 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 07 13:02:01.647992 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.647676 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qnpdc\"" Sep 07 13:02:01.652923 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.652899 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fwckf"] Sep 07 13:02:01.654068 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.654001 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w"] Sep 07 13:02:01.729982 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.729934 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c7991f8f-dd41-437e-8d24-8d2b3c756304-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-2qm2w\" (UID: \"c7991f8f-dd41-437e-8d24-8d2b3c756304\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w" Sep 07 13:02:01.730188 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.730032 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c7991f8f-dd41-437e-8d24-8d2b3c756304-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-2qm2w\" (UID: \"c7991f8f-dd41-437e-8d24-8d2b3c756304\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w" Sep 07 13:02:01.730188 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.730096 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgdgz\" (UniqueName: \"kubernetes.io/projected/c7991f8f-dd41-437e-8d24-8d2b3c756304-kube-api-access-rgdgz\") pod \"kube-storage-version-migrator-operator-8d7dd4675-2qm2w\" (UID: \"c7991f8f-dd41-437e-8d24-8d2b3c756304\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w" Sep 07 13:02:01.730188 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.730127 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkr6m\" (UniqueName: \"kubernetes.io/projected/2313840a-af08-4019-a75c-01db10b0df66-kube-api-access-rkr6m\") pod \"volume-data-source-validator-695c895bd6-fwckf\" (UID: \"2313840a-af08-4019-a75c-01db10b0df66\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fwckf" Sep 07 13:02:01.735347 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.735319 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-74489cdf66-fv7gs"] Sep 07 13:02:01.738161 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.738144 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:01.741226 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.741191 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5vldd\"" Sep 07 13:02:01.741370 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.741194 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 07 13:02:01.741608 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.741579 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 07 13:02:01.741719 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.741620 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 07 13:02:01.741931 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.741913 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 07 13:02:01.742091 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.742046 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 07 13:02:01.742389 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.742254 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 07 13:02:01.756516 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.756490 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74489cdf66-fv7gs"] Sep 07 13:02:01.831166 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.831124 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgdgz\" (UniqueName: \"kubernetes.io/projected/c7991f8f-dd41-437e-8d24-8d2b3c756304-kube-api-access-rgdgz\") pod \"kube-storage-version-migrator-operator-8d7dd4675-2qm2w\" (UID: \"c7991f8f-dd41-437e-8d24-8d2b3c756304\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w" Sep 07 13:02:01.831166 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.831169 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkr6m\" (UniqueName: \"kubernetes.io/projected/2313840a-af08-4019-a75c-01db10b0df66-kube-api-access-rkr6m\") pod \"volume-data-source-validator-695c895bd6-fwckf\" (UID: \"2313840a-af08-4019-a75c-01db10b0df66\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fwckf" Sep 07 13:02:01.831382 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.831196 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-default-certificate\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:01.831382 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.831218 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scxgd\" (UniqueName: \"kubernetes.io/projected/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-kube-api-access-scxgd\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:01.831382 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.831284 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:01.831382 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.831313 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-stats-auth\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:01.831382 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.831339 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c7991f8f-dd41-437e-8d24-8d2b3c756304-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-2qm2w\" (UID: \"c7991f8f-dd41-437e-8d24-8d2b3c756304\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w" Sep 07 13:02:01.831586 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.831402 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c7991f8f-dd41-437e-8d24-8d2b3c756304-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-2qm2w\" (UID: \"c7991f8f-dd41-437e-8d24-8d2b3c756304\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w" Sep 07 13:02:01.831586 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.831417 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:01.831879 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.831858 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c7991f8f-dd41-437e-8d24-8d2b3c756304-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-2qm2w\" (UID: \"c7991f8f-dd41-437e-8d24-8d2b3c756304\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w" Sep 07 13:02:01.833584 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.833563 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c7991f8f-dd41-437e-8d24-8d2b3c756304-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-2qm2w\" (UID: \"c7991f8f-dd41-437e-8d24-8d2b3c756304\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w" Sep 07 13:02:01.839983 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.839955 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgdgz\" (UniqueName: \"kubernetes.io/projected/c7991f8f-dd41-437e-8d24-8d2b3c756304-kube-api-access-rgdgz\") pod \"kube-storage-version-migrator-operator-8d7dd4675-2qm2w\" (UID: \"c7991f8f-dd41-437e-8d24-8d2b3c756304\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w" Sep 07 13:02:01.840159 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.840137 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkr6m\" (UniqueName: \"kubernetes.io/projected/2313840a-af08-4019-a75c-01db10b0df66-kube-api-access-rkr6m\") pod \"volume-data-source-validator-695c895bd6-fwckf\" (UID: \"2313840a-af08-4019-a75c-01db10b0df66\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fwckf" Sep 07 13:02:01.932694 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.932606 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-stats-auth\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:01.932694 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.932677 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:01.932928 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.932708 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-default-certificate\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:01.932928 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.932724 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scxgd\" (UniqueName: \"kubernetes.io/projected/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-kube-api-access-scxgd\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:01.932928 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.932745 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:01.932928 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:01.932813 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle podName:790dc1f9-f44a-4c9d-a081-8de58dd6d08e nodeName:}" failed. No retries permitted until 2026-09-07 13:02:02.432795179 +0000 UTC m=+83.028299789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle") pod "router-default-74489cdf66-fv7gs" (UID: "790dc1f9-f44a-4c9d-a081-8de58dd6d08e") : configmap references non-existent config key: service-ca.crt Sep 07 13:02:01.932928 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:01.932822 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 13:02:01.932928 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:01.932900 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs podName:790dc1f9-f44a-4c9d-a081-8de58dd6d08e nodeName:}" failed. No retries permitted until 2026-09-07 13:02:02.432885095 +0000 UTC m=+83.028389718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs") pod "router-default-74489cdf66-fv7gs" (UID: "790dc1f9-f44a-4c9d-a081-8de58dd6d08e") : secret "router-metrics-certs-default" not found Sep 07 13:02:01.935077 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.935042 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-stats-auth\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:01.935171 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.935097 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-default-certificate\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:01.944066 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.944033 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scxgd\" (UniqueName: \"kubernetes.io/projected/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-kube-api-access-scxgd\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:01.950666 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.950649 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fwckf" Sep 07 13:02:01.956538 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:01.956524 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w" Sep 07 13:02:02.228637 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:02.228605 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fwckf"] Sep 07 13:02:02.232255 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:02:02.232226 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2313840a_af08_4019_a75c_01db10b0df66.slice/crio-d13f7fea7bbcf4d58b95dc3f98d393f08c4c9ef2203208f75ff7350d60a74f84 WatchSource:0}: Error finding container d13f7fea7bbcf4d58b95dc3f98d393f08c4c9ef2203208f75ff7350d60a74f84: Status 404 returned error can't find the container with id d13f7fea7bbcf4d58b95dc3f98d393f08c4c9ef2203208f75ff7350d60a74f84 Sep 07 13:02:02.232255 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:02.232236 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w"] Sep 07 13:02:02.235092 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:02:02.235047 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7991f8f_dd41_437e_8d24_8d2b3c756304.slice/crio-ac927a35985ad8080b53fe90916681ba2bf5639a9d3a11580222ba17f9ce32e2 WatchSource:0}: Error finding container ac927a35985ad8080b53fe90916681ba2bf5639a9d3a11580222ba17f9ce32e2: Status 404 returned error can't find the container with id ac927a35985ad8080b53fe90916681ba2bf5639a9d3a11580222ba17f9ce32e2 Sep 07 13:02:02.436957 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:02.436908 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:02.436957 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:02.436975 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:02.437203 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:02.437081 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 13:02:02.437203 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:02.437120 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle podName:790dc1f9-f44a-4c9d-a081-8de58dd6d08e nodeName:}" failed. No retries permitted until 2026-09-07 13:02:03.437099799 +0000 UTC m=+84.032604393 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle") pod "router-default-74489cdf66-fv7gs" (UID: "790dc1f9-f44a-4c9d-a081-8de58dd6d08e") : configmap references non-existent config key: service-ca.crt Sep 07 13:02:02.437203 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:02.437147 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs podName:790dc1f9-f44a-4c9d-a081-8de58dd6d08e nodeName:}" failed. No retries permitted until 2026-09-07 13:02:03.437139486 +0000 UTC m=+84.032644080 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs") pod "router-default-74489cdf66-fv7gs" (UID: "790dc1f9-f44a-4c9d-a081-8de58dd6d08e") : secret "router-metrics-certs-default" not found Sep 07 13:02:03.224313 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:03.224278 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w" event={"ID":"c7991f8f-dd41-437e-8d24-8d2b3c756304","Type":"ContainerStarted","Data":"ac927a35985ad8080b53fe90916681ba2bf5639a9d3a11580222ba17f9ce32e2"} Sep 07 13:02:03.225547 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:03.225523 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fwckf" event={"ID":"2313840a-af08-4019-a75c-01db10b0df66","Type":"ContainerStarted","Data":"d13f7fea7bbcf4d58b95dc3f98d393f08c4c9ef2203208f75ff7350d60a74f84"} Sep 07 13:02:03.444559 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:03.444511 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:03.444806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:03.444644 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:03.444806 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:03.444662 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 13:02:03.444806 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:03.444734 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs podName:790dc1f9-f44a-4c9d-a081-8de58dd6d08e nodeName:}" failed. No retries permitted until 2026-09-07 13:02:05.444711317 +0000 UTC m=+86.040215934 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs") pod "router-default-74489cdf66-fv7gs" (UID: "790dc1f9-f44a-4c9d-a081-8de58dd6d08e") : secret "router-metrics-certs-default" not found Sep 07 13:02:03.444806 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:03.444771 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle podName:790dc1f9-f44a-4c9d-a081-8de58dd6d08e nodeName:}" failed. No retries permitted until 2026-09-07 13:02:05.444755937 +0000 UTC m=+86.040260549 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle") pod "router-default-74489cdf66-fv7gs" (UID: "790dc1f9-f44a-4c9d-a081-8de58dd6d08e") : configmap references non-existent config key: service-ca.crt Sep 07 13:02:04.230384 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:04.230346 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fwckf" event={"ID":"2313840a-af08-4019-a75c-01db10b0df66","Type":"ContainerStarted","Data":"ec67aeb90beacee9a3292b8d2fc1e48e5a78db9f2f4f68d8c0a6c1b0f1289b08"} Sep 07 13:02:04.253537 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:04.253445 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fwckf" podStartSLOduration=2.243914053 podStartE2EDuration="3.25342616s" totalImagesPullingTime="1.009512107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:02:04.253170126 +0000 UTC m=+84.848674736" watchObservedRunningTime="2026-09-07 13:02:04.25342616 +0000 UTC m=+84.848930779" Sep 07 13:02:05.233861 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:05.233824 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w" event={"ID":"c7991f8f-dd41-437e-8d24-8d2b3c756304","Type":"ContainerStarted","Data":"c7e44c455378af2eb7f743d8cfedb8c79b39509079847b5fae0aa6573e9c7cf5"} Sep 07 13:02:05.257947 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:05.257898 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w" podStartSLOduration=2.204206269 podStartE2EDuration="4.257881153s" totalImagesPullingTime="2.053674884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:02:05.256823335 +0000 UTC m=+85.852327962" watchObservedRunningTime="2026-09-07 13:02:05.257881153 +0000 UTC m=+85.853385768" Sep 07 13:02:05.461911 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:05.461874 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:05.462083 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:05.461924 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:05.462083 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:05.462031 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 13:02:05.462163 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:05.462082 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle podName:790dc1f9-f44a-4c9d-a081-8de58dd6d08e nodeName:}" failed. No retries permitted until 2026-09-07 13:02:09.462062176 +0000 UTC m=+90.057566783 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle") pod "router-default-74489cdf66-fv7gs" (UID: "790dc1f9-f44a-4c9d-a081-8de58dd6d08e") : configmap references non-existent config key: service-ca.crt Sep 07 13:02:05.462163 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:05.462127 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs podName:790dc1f9-f44a-4c9d-a081-8de58dd6d08e nodeName:}" failed. No retries permitted until 2026-09-07 13:02:09.46210013 +0000 UTC m=+90.057604724 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs") pod "router-default-74489cdf66-fv7gs" (UID: "790dc1f9-f44a-4c9d-a081-8de58dd6d08e") : secret "router-metrics-certs-default" not found Sep 07 13:02:06.847218 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:06.847185 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mkg7m"] Sep 07 13:02:06.850397 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:06.850380 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:06.854267 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:06.854248 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 07 13:02:06.854451 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:06.854435 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 07 13:02:06.854521 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:06.854456 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 07 13:02:06.854521 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:06.854469 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 07 13:02:06.854646 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:06.854463 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gzntm\"" Sep 07 13:02:06.868372 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:06.868347 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mkg7m"] Sep 07 13:02:06.974848 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:06.974803 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/845ff0c5-eab8-4c81-abca-19f574c1dcdb-crio-socket\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:06.975003 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:06.974884 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/845ff0c5-eab8-4c81-abca-19f574c1dcdb-data-volume\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:06.975003 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:06.974929 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:06.975003 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:06.974953 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:06.975175 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:06.975094 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4v76\" (UniqueName: \"kubernetes.io/projected/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-api-access-w4v76\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:07.076194 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:07.076161 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/845ff0c5-eab8-4c81-abca-19f574c1dcdb-crio-socket\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:07.076194 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:07.076194 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/845ff0c5-eab8-4c81-abca-19f574c1dcdb-data-volume\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:07.076406 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:07.076209 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:07.076406 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:07.076226 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:07.076406 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:07.076290 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/845ff0c5-eab8-4c81-abca-19f574c1dcdb-crio-socket\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:07.076406 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:07.076327 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 13:02:07.076406 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:07.076348 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 13:02:07.076406 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:07.076395 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm podName:845ff0c5-eab8-4c81-abca-19f574c1dcdb nodeName:}" failed. No retries permitted until 2026-09-07 13:02:07.57637445 +0000 UTC m=+88.171879057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mkg7m" (UID: "845ff0c5-eab8-4c81-abca-19f574c1dcdb") : configmap "kube-rbac-proxy" not found Sep 07 13:02:07.076694 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:07.076413 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls podName:845ff0c5-eab8-4c81-abca-19f574c1dcdb nodeName:}" failed. No retries permitted until 2026-09-07 13:02:07.576404797 +0000 UTC m=+88.171909417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mkg7m" (UID: "845ff0c5-eab8-4c81-abca-19f574c1dcdb") : secret "insights-runtime-extractor-tls" not found Sep 07 13:02:07.076694 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:07.076451 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4v76\" (UniqueName: \"kubernetes.io/projected/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-api-access-w4v76\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:07.076694 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:07.076557 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/845ff0c5-eab8-4c81-abca-19f574c1dcdb-data-volume\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:07.086717 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:07.086690 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4v76\" (UniqueName: \"kubernetes.io/projected/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-api-access-w4v76\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:07.580547 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:07.580494 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:07.580547 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:07.580552 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:07.580819 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:07.580636 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 13:02:07.580819 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:07.580698 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 13:02:07.580819 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:07.580715 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm podName:845ff0c5-eab8-4c81-abca-19f574c1dcdb nodeName:}" failed. No retries permitted until 2026-09-07 13:02:08.580699259 +0000 UTC m=+89.176203853 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mkg7m" (UID: "845ff0c5-eab8-4c81-abca-19f574c1dcdb") : configmap "kube-rbac-proxy" not found Sep 07 13:02:07.580819 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:07.580751 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls podName:845ff0c5-eab8-4c81-abca-19f574c1dcdb nodeName:}" failed. No retries permitted until 2026-09-07 13:02:08.580735655 +0000 UTC m=+89.176240264 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mkg7m" (UID: "845ff0c5-eab8-4c81-abca-19f574c1dcdb") : secret "insights-runtime-extractor-tls" not found Sep 07 13:02:08.589772 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:08.589731 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:08.589772 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:08.589772 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:08.590339 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:08.589853 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 13:02:08.590339 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:08.589874 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 13:02:08.590339 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:08.589925 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm podName:845ff0c5-eab8-4c81-abca-19f574c1dcdb nodeName:}" failed. No retries permitted until 2026-09-07 13:02:10.589908599 +0000 UTC m=+91.185413219 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mkg7m" (UID: "845ff0c5-eab8-4c81-abca-19f574c1dcdb") : configmap "kube-rbac-proxy" not found Sep 07 13:02:08.590339 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:08.589943 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls podName:845ff0c5-eab8-4c81-abca-19f574c1dcdb nodeName:}" failed. No retries permitted until 2026-09-07 13:02:10.589934796 +0000 UTC m=+91.185439439 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mkg7m" (UID: "845ff0c5-eab8-4c81-abca-19f574c1dcdb") : secret "insights-runtime-extractor-tls" not found Sep 07 13:02:09.497148 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:09.497107 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:09.497298 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:09.497192 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:09.497365 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:09.497291 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle podName:790dc1f9-f44a-4c9d-a081-8de58dd6d08e nodeName:}" failed. No retries permitted until 2026-09-07 13:02:17.497272244 +0000 UTC m=+98.092776857 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle") pod "router-default-74489cdf66-fv7gs" (UID: "790dc1f9-f44a-4c9d-a081-8de58dd6d08e") : configmap references non-existent config key: service-ca.crt Sep 07 13:02:09.497365 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:09.497312 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 13:02:09.497365 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:09.497348 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs podName:790dc1f9-f44a-4c9d-a081-8de58dd6d08e nodeName:}" failed. No retries permitted until 2026-09-07 13:02:17.497338294 +0000 UTC m=+98.092842887 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs") pod "router-default-74489cdf66-fv7gs" (UID: "790dc1f9-f44a-4c9d-a081-8de58dd6d08e") : secret "router-metrics-certs-default" not found Sep 07 13:02:09.594641 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:09.594613 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fcd4k_912823fa-6c24-493a-8640-89b921cdf2f2/dns-node-resolver/0.log" Sep 07 13:02:10.594420 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:10.594391 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hrgm6_ec3637df-91b4-4cbe-8d8b-7378b2c12a4f/node-ca/0.log" Sep 07 13:02:10.604720 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:10.604691 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:10.605177 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:10.604737 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:10.605177 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:10.604766 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 13:02:10.605177 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:10.604824 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 13:02:10.605177 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:10.604852 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm podName:845ff0c5-eab8-4c81-abca-19f574c1dcdb nodeName:}" failed. No retries permitted until 2026-09-07 13:02:14.604831549 +0000 UTC m=+95.200336164 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mkg7m" (UID: "845ff0c5-eab8-4c81-abca-19f574c1dcdb") : configmap "kube-rbac-proxy" not found Sep 07 13:02:10.605177 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:10.604873 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls podName:845ff0c5-eab8-4c81-abca-19f574c1dcdb nodeName:}" failed. No retries permitted until 2026-09-07 13:02:14.604862778 +0000 UTC m=+95.200367371 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mkg7m" (UID: "845ff0c5-eab8-4c81-abca-19f574c1dcdb") : secret "insights-runtime-extractor-tls" not found Sep 07 13:02:11.202145 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:11.202112 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6gpjx_3f13b2c1-d6fb-4722-b60e-4b940d617447/serve-healthcheck-canary/0.log" Sep 07 13:02:12.196395 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:12.196364 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-2qm2w_c7991f8f-dd41-437e-8d24-8d2b3c756304/kube-storage-version-migrator-operator/0.log" Sep 07 13:02:14.637951 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:14.637914 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:14.637951 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:14.637954 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:14.638525 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:14.638044 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 13:02:14.638525 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:14.638080 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 13:02:14.638525 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:14.638133 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls podName:845ff0c5-eab8-4c81-abca-19f574c1dcdb nodeName:}" failed. No retries permitted until 2026-09-07 13:02:22.638119538 +0000 UTC m=+103.233624131 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mkg7m" (UID: "845ff0c5-eab8-4c81-abca-19f574c1dcdb") : secret "insights-runtime-extractor-tls" not found Sep 07 13:02:14.638525 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:14.638146 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm podName:845ff0c5-eab8-4c81-abca-19f574c1dcdb nodeName:}" failed. No retries permitted until 2026-09-07 13:02:22.638140509 +0000 UTC m=+103.233645102 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mkg7m" (UID: "845ff0c5-eab8-4c81-abca-19f574c1dcdb") : configmap "kube-rbac-proxy" not found Sep 07 13:02:17.560683 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:17.560645 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:17.561128 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:17.560713 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:17.561128 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:17.560812 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 07 13:02:17.561128 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:17.560820 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle podName:790dc1f9-f44a-4c9d-a081-8de58dd6d08e nodeName:}" failed. No retries permitted until 2026-09-07 13:02:33.560801817 +0000 UTC m=+114.156306430 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle") pod "router-default-74489cdf66-fv7gs" (UID: "790dc1f9-f44a-4c9d-a081-8de58dd6d08e") : configmap references non-existent config key: service-ca.crt Sep 07 13:02:17.561128 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:17.560862 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs podName:790dc1f9-f44a-4c9d-a081-8de58dd6d08e nodeName:}" failed. No retries permitted until 2026-09-07 13:02:33.560851365 +0000 UTC m=+114.156355960 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs") pod "router-default-74489cdf66-fv7gs" (UID: "790dc1f9-f44a-4c9d-a081-8de58dd6d08e") : secret "router-metrics-certs-default" not found Sep 07 13:02:18.670614 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:18.670524 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:02:18.670614 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:18.670585 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:02:18.672912 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:18.672879 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66d7fa5d-e887-46a9-b527-a7a8398e4786-metrics-tls\") pod \"dns-default-74wz2\" (UID: \"66d7fa5d-e887-46a9-b527-a7a8398e4786\") " pod="openshift-dns/dns-default-74wz2" Sep 07 13:02:18.673042 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:18.672958 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls\") pod \"image-registry-864459d56-bbg5b\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:02:18.691679 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:18.691655 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fsnx6\"" Sep 07 13:02:18.699795 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:18.699774 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:02:18.707551 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:18.707528 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-77fxd\"" Sep 07 13:02:18.715173 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:18.715147 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-74wz2" Sep 07 13:02:18.993073 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:18.993027 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-864459d56-bbg5b"] Sep 07 13:02:18.996188 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:02:18.996163 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b87e5f6_90fc_4891_aa11_feb499ce5511.slice/crio-2fb41ab912fbaa3da9ad28b0424b673551714aa727a0e3c5048e34c2722993bd WatchSource:0}: Error finding container 2fb41ab912fbaa3da9ad28b0424b673551714aa727a0e3c5048e34c2722993bd: Status 404 returned error can't find the container with id 2fb41ab912fbaa3da9ad28b0424b673551714aa727a0e3c5048e34c2722993bd Sep 07 13:02:19.015477 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:19.015452 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-74wz2"] Sep 07 13:02:19.025003 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:02:19.024978 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66d7fa5d_e887_46a9_b527_a7a8398e4786.slice/crio-8fa2c4234b7656c39fcc3f58eb9bec694b504d78048c6b081f28635418693909 WatchSource:0}: Error finding container 8fa2c4234b7656c39fcc3f58eb9bec694b504d78048c6b081f28635418693909: Status 404 returned error can't find the container with id 8fa2c4234b7656c39fcc3f58eb9bec694b504d78048c6b081f28635418693909 Sep 07 13:02:19.271008 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:19.270908 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-74wz2" event={"ID":"66d7fa5d-e887-46a9-b527-a7a8398e4786","Type":"ContainerStarted","Data":"8fa2c4234b7656c39fcc3f58eb9bec694b504d78048c6b081f28635418693909"} Sep 07 13:02:19.272440 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:19.272414 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864459d56-bbg5b" event={"ID":"4b87e5f6-90fc-4891-aa11-feb499ce5511","Type":"ContainerStarted","Data":"4ac9a49d7df276c2d12dd7d2c274a1dcbab89791c6147576d8dfbb18f2045cf1"} Sep 07 13:02:19.272559 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:19.272452 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864459d56-bbg5b" event={"ID":"4b87e5f6-90fc-4891-aa11-feb499ce5511","Type":"ContainerStarted","Data":"2fb41ab912fbaa3da9ad28b0424b673551714aa727a0e3c5048e34c2722993bd"} Sep 07 13:02:19.272559 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:19.272549 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:02:19.298703 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:19.298651 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-864459d56-bbg5b" podStartSLOduration=98.298632769 podStartE2EDuration="1m38.298632769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:00:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:02:19.297822554 +0000 UTC m=+99.893327170" watchObservedRunningTime="2026-09-07 13:02:19.298632769 +0000 UTC m=+99.894137385" Sep 07 13:02:21.281302 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:21.281266 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-74wz2" event={"ID":"66d7fa5d-e887-46a9-b527-a7a8398e4786","Type":"ContainerStarted","Data":"73ff63593420967ff0ed799a83e0a41749eeafff28d71dca4e631c30a96e2920"} Sep 07 13:02:21.281777 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:21.281320 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-74wz2" event={"ID":"66d7fa5d-e887-46a9-b527-a7a8398e4786","Type":"ContainerStarted","Data":"53aeaa339c4fb9b351b0a6e5267c776467916967bd488e40b377117927314acb"} Sep 07 13:02:21.281777 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:21.281405 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-74wz2" Sep 07 13:02:21.299529 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:21.299478 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-74wz2" podStartSLOduration=65.646239001 podStartE2EDuration="1m7.299462206s" totalImagesPullingTime="1.653223205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:02:21.298696172 +0000 UTC m=+101.894200789" watchObservedRunningTime="2026-09-07 13:02:21.299462206 +0000 UTC m=+101.894966866" Sep 07 13:02:22.200098 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:22.200066 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-plvr8" Sep 07 13:02:22.709035 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:22.708975 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:22.709035 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:22.709037 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:22.709616 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:22.709168 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 13:02:22.709616 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:22.709246 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls podName:845ff0c5-eab8-4c81-abca-19f574c1dcdb nodeName:}" failed. No retries permitted until 2026-09-07 13:02:38.70922829 +0000 UTC m=+119.304732905 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mkg7m" (UID: "845ff0c5-eab8-4c81-abca-19f574c1dcdb") : secret "insights-runtime-extractor-tls" not found Sep 07 13:02:22.709616 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:22.709508 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/845ff0c5-eab8-4c81-abca-19f574c1dcdb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:31.286344 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:31.286310 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-74wz2" Sep 07 13:02:32.692126 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:32.692078 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8"] Sep 07 13:02:32.694560 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:32.694514 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8" Sep 07 13:02:32.697528 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:32.697506 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 07 13:02:32.697631 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:32.697545 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 07 13:02:32.697631 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:32.697506 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 07 13:02:32.697631 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:32.697624 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 07 13:02:32.706283 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:32.706258 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8"] Sep 07 13:02:32.790479 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:32.790422 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6532c603-df2d-4d2f-87d9-055ecf3a6810-klusterlet-config\") pod \"klusterlet-addon-workmgr-7bbbb4d965-d6mz8\" (UID: \"6532c603-df2d-4d2f-87d9-055ecf3a6810\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8" Sep 07 13:02:32.790648 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:32.790556 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6532c603-df2d-4d2f-87d9-055ecf3a6810-tmp\") pod \"klusterlet-addon-workmgr-7bbbb4d965-d6mz8\" (UID: \"6532c603-df2d-4d2f-87d9-055ecf3a6810\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8" Sep 07 13:02:32.790648 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:32.790613 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkpzq\" (UniqueName: \"kubernetes.io/projected/6532c603-df2d-4d2f-87d9-055ecf3a6810-kube-api-access-rkpzq\") pod \"klusterlet-addon-workmgr-7bbbb4d965-d6mz8\" (UID: \"6532c603-df2d-4d2f-87d9-055ecf3a6810\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8" Sep 07 13:02:32.891228 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:32.891179 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6532c603-df2d-4d2f-87d9-055ecf3a6810-klusterlet-config\") pod \"klusterlet-addon-workmgr-7bbbb4d965-d6mz8\" (UID: \"6532c603-df2d-4d2f-87d9-055ecf3a6810\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8" Sep 07 13:02:32.891431 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:32.891241 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6532c603-df2d-4d2f-87d9-055ecf3a6810-tmp\") pod \"klusterlet-addon-workmgr-7bbbb4d965-d6mz8\" (UID: \"6532c603-df2d-4d2f-87d9-055ecf3a6810\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8" Sep 07 13:02:32.891431 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:32.891276 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkpzq\" (UniqueName: \"kubernetes.io/projected/6532c603-df2d-4d2f-87d9-055ecf3a6810-kube-api-access-rkpzq\") pod \"klusterlet-addon-workmgr-7bbbb4d965-d6mz8\" (UID: \"6532c603-df2d-4d2f-87d9-055ecf3a6810\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8" Sep 07 13:02:32.891734 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:32.891710 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6532c603-df2d-4d2f-87d9-055ecf3a6810-tmp\") pod \"klusterlet-addon-workmgr-7bbbb4d965-d6mz8\" (UID: \"6532c603-df2d-4d2f-87d9-055ecf3a6810\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8" Sep 07 13:02:32.894006 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:32.893968 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6532c603-df2d-4d2f-87d9-055ecf3a6810-klusterlet-config\") pod \"klusterlet-addon-workmgr-7bbbb4d965-d6mz8\" (UID: \"6532c603-df2d-4d2f-87d9-055ecf3a6810\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8" Sep 07 13:02:32.900152 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:32.900131 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkpzq\" (UniqueName: \"kubernetes.io/projected/6532c603-df2d-4d2f-87d9-055ecf3a6810-kube-api-access-rkpzq\") pod \"klusterlet-addon-workmgr-7bbbb4d965-d6mz8\" (UID: \"6532c603-df2d-4d2f-87d9-055ecf3a6810\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8" Sep 07 13:02:33.003269 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:33.003241 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8" Sep 07 13:02:33.273369 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:33.273295 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8"] Sep 07 13:02:33.276682 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:02:33.276651 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6532c603_df2d_4d2f_87d9_055ecf3a6810.slice/crio-ffe26f5de19e558fcc812f63150f548626d3574374c5947b75b511b3297dab9d WatchSource:0}: Error finding container ffe26f5de19e558fcc812f63150f548626d3574374c5947b75b511b3297dab9d: Status 404 returned error can't find the container with id ffe26f5de19e558fcc812f63150f548626d3574374c5947b75b511b3297dab9d Sep 07 13:02:33.319982 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:33.319944 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8" event={"ID":"6532c603-df2d-4d2f-87d9-055ecf3a6810","Type":"ContainerStarted","Data":"ffe26f5de19e558fcc812f63150f548626d3574374c5947b75b511b3297dab9d"} Sep 07 13:02:33.605147 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:33.605035 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:33.605147 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:33.605112 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:33.605675 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:33.605654 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-service-ca-bundle\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:33.607462 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:33.607438 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/790dc1f9-f44a-4c9d-a081-8de58dd6d08e-metrics-certs\") pod \"router-default-74489cdf66-fv7gs\" (UID: \"790dc1f9-f44a-4c9d-a081-8de58dd6d08e\") " pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:33.847622 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:33.847585 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:34.129515 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:34.129299 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74489cdf66-fv7gs"] Sep 07 13:02:34.133863 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:02:34.133836 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod790dc1f9_f44a_4c9d_a081_8de58dd6d08e.slice/crio-edf2f0fb2b7f20c62688774af1faea0332644313d1b794391b07328582b1e875 WatchSource:0}: Error finding container edf2f0fb2b7f20c62688774af1faea0332644313d1b794391b07328582b1e875: Status 404 returned error can't find the container with id edf2f0fb2b7f20c62688774af1faea0332644313d1b794391b07328582b1e875 Sep 07 13:02:34.326152 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:34.326101 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74489cdf66-fv7gs" event={"ID":"790dc1f9-f44a-4c9d-a081-8de58dd6d08e","Type":"ContainerStarted","Data":"2ca959f8ab9ce4b9e03d1bda36da25283dbc704a7f8280f347f91aa6ddab7fd6"} Sep 07 13:02:34.326382 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:34.326166 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74489cdf66-fv7gs" event={"ID":"790dc1f9-f44a-4c9d-a081-8de58dd6d08e","Type":"ContainerStarted","Data":"edf2f0fb2b7f20c62688774af1faea0332644313d1b794391b07328582b1e875"} Sep 07 13:02:34.352854 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:34.352787 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-74489cdf66-fv7gs" podStartSLOduration=33.352766793 podStartE2EDuration="33.352766793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:02:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:02:34.35193924 +0000 UTC m=+114.947443871" watchObservedRunningTime="2026-09-07 13:02:34.352766793 +0000 UTC m=+114.948271413" Sep 07 13:02:34.848024 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:34.847979 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:34.851389 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:34.851358 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:35.330066 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:35.330021 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:35.331433 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:35.331401 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-74489cdf66-fv7gs" Sep 07 13:02:37.337493 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:37.337452 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8" event={"ID":"6532c603-df2d-4d2f-87d9-055ecf3a6810","Type":"ContainerStarted","Data":"ebc416f2e0ed034922fbb0125f93bde08c0c216aa490c8c7ec8124b2f63d6e5b"} Sep 07 13:02:37.337899 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:37.337785 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8" Sep 07 13:02:37.339301 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:37.339280 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8" Sep 07 13:02:37.357465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:37.357415 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbbb4d965-d6mz8" podStartSLOduration=1.446885581 podStartE2EDuration="5.357401038s" totalImagesPullingTime="3.910515457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:02:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:02:37.355692037 +0000 UTC m=+117.951196654" watchObservedRunningTime="2026-09-07 13:02:37.357401038 +0000 UTC m=+117.952905657" Sep 07 13:02:37.427292 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:37.427254 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nkfjl"] Sep 07 13:02:37.428926 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:37.428909 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nkfjl" Sep 07 13:02:37.431851 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:37.431829 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 07 13:02:37.431968 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:37.431905 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-l62c5\"" Sep 07 13:02:37.440703 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:37.440645 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nkfjl"] Sep 07 13:02:37.441149 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:37.441040 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/29d0b5ca-7cd9-49fd-b14f-4f16ffbe67bc-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-nkfjl\" (UID: \"29d0b5ca-7cd9-49fd-b14f-4f16ffbe67bc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nkfjl" Sep 07 13:02:37.541994 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:37.541955 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/29d0b5ca-7cd9-49fd-b14f-4f16ffbe67bc-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-nkfjl\" (UID: \"29d0b5ca-7cd9-49fd-b14f-4f16ffbe67bc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nkfjl" Sep 07 13:02:37.544435 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:37.544377 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/29d0b5ca-7cd9-49fd-b14f-4f16ffbe67bc-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-nkfjl\" (UID: \"29d0b5ca-7cd9-49fd-b14f-4f16ffbe67bc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nkfjl" Sep 07 13:02:37.737295 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:37.737254 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nkfjl" Sep 07 13:02:38.007705 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:38.007618 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nkfjl"] Sep 07 13:02:38.010576 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:02:38.010550 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29d0b5ca_7cd9_49fd_b14f_4f16ffbe67bc.slice/crio-424adfa10f859d0d5d176ecb53c241cd62c6db717099f35ab626e8cdfe507bdb WatchSource:0}: Error finding container 424adfa10f859d0d5d176ecb53c241cd62c6db717099f35ab626e8cdfe507bdb: Status 404 returned error can't find the container with id 424adfa10f859d0d5d176ecb53c241cd62c6db717099f35ab626e8cdfe507bdb Sep 07 13:02:38.341700 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:38.341599 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nkfjl" event={"ID":"29d0b5ca-7cd9-49fd-b14f-4f16ffbe67bc","Type":"ContainerStarted","Data":"424adfa10f859d0d5d176ecb53c241cd62c6db717099f35ab626e8cdfe507bdb"} Sep 07 13:02:38.704573 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:38.704489 2564 patch_prober.go:28] interesting pod/image-registry-864459d56-bbg5b container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 07 13:02:38.704573 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:38.704546 2564 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-864459d56-bbg5b" podUID="4b87e5f6-90fc-4891-aa11-feb499ce5511" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 07 13:02:38.751117 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:38.751082 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:38.753308 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:38.753279 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/845ff0c5-eab8-4c81-abca-19f574c1dcdb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mkg7m\" (UID: \"845ff0c5-eab8-4c81-abca-19f574c1dcdb\") " pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:38.959600 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:38.959516 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mkg7m" Sep 07 13:02:39.236129 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:39.236095 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mkg7m"] Sep 07 13:02:39.302172 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:02:39.302142 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod845ff0c5_eab8_4c81_abca_19f574c1dcdb.slice/crio-9922af1eff2586353c29e94bc1e2f68c02897867c8b617fe6604557ffdf0740d WatchSource:0}: Error finding container 9922af1eff2586353c29e94bc1e2f68c02897867c8b617fe6604557ffdf0740d: Status 404 returned error can't find the container with id 9922af1eff2586353c29e94bc1e2f68c02897867c8b617fe6604557ffdf0740d Sep 07 13:02:39.346374 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:39.346344 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mkg7m" event={"ID":"845ff0c5-eab8-4c81-abca-19f574c1dcdb","Type":"ContainerStarted","Data":"9922af1eff2586353c29e94bc1e2f68c02897867c8b617fe6604557ffdf0740d"} Sep 07 13:02:40.280824 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:40.280791 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:02:40.352223 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:40.352176 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mkg7m" event={"ID":"845ff0c5-eab8-4c81-abca-19f574c1dcdb","Type":"ContainerStarted","Data":"0066559fbbb5b22b6079973459a5e00cab615732d9d2791595a243d188f05331"} Sep 07 13:02:40.352658 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:40.352242 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mkg7m" event={"ID":"845ff0c5-eab8-4c81-abca-19f574c1dcdb","Type":"ContainerStarted","Data":"94cdd57baa31bc1703f5bed57affbe974b3ce7c81ebf192681f293091dd25a16"} Sep 07 13:02:40.353934 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:40.353908 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nkfjl" event={"ID":"29d0b5ca-7cd9-49fd-b14f-4f16ffbe67bc","Type":"ContainerStarted","Data":"72b1dbe77bd5856c06bb8cb8bcfa3d0842a629e0d491b21d3c9740628659a77c"} Sep 07 13:02:40.354239 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:40.354170 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nkfjl" Sep 07 13:02:40.358596 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:40.358575 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nkfjl" Sep 07 13:02:40.370385 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:40.370336 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nkfjl" podStartSLOduration=2.060303798 podStartE2EDuration="3.370324691s" totalImagesPullingTime="1.310020893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:02:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:02:40.370242768 +0000 UTC m=+120.965747384" watchObservedRunningTime="2026-09-07 13:02:40.370324691 +0000 UTC m=+120.965829308" Sep 07 13:02:41.508907 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.508874 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-lj5t6"] Sep 07 13:02:41.510874 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.510845 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" Sep 07 13:02:41.513342 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.513315 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 07 13:02:41.514315 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.514291 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 07 13:02:41.514424 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.514330 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 07 13:02:41.514424 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.514405 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 07 13:02:41.514641 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.514623 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 07 13:02:41.514710 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.514672 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bdrhk\"" Sep 07 13:02:41.522613 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.522592 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-lj5t6"] Sep 07 13:02:41.575462 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.575414 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0aca2db-7f85-4df0-8440-ad3e9767d3c9-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-lj5t6\" (UID: \"a0aca2db-7f85-4df0-8440-ad3e9767d3c9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" Sep 07 13:02:41.575658 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.575495 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0aca2db-7f85-4df0-8440-ad3e9767d3c9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-lj5t6\" (UID: \"a0aca2db-7f85-4df0-8440-ad3e9767d3c9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" Sep 07 13:02:41.575658 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.575554 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a0aca2db-7f85-4df0-8440-ad3e9767d3c9-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-lj5t6\" (UID: \"a0aca2db-7f85-4df0-8440-ad3e9767d3c9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" Sep 07 13:02:41.575658 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.575632 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fjdp\" (UniqueName: \"kubernetes.io/projected/a0aca2db-7f85-4df0-8440-ad3e9767d3c9-kube-api-access-4fjdp\") pod \"prometheus-operator-6f4946db56-lj5t6\" (UID: \"a0aca2db-7f85-4df0-8440-ad3e9767d3c9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" Sep 07 13:02:41.676502 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.676463 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0aca2db-7f85-4df0-8440-ad3e9767d3c9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-lj5t6\" (UID: \"a0aca2db-7f85-4df0-8440-ad3e9767d3c9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" Sep 07 13:02:41.676756 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.676510 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a0aca2db-7f85-4df0-8440-ad3e9767d3c9-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-lj5t6\" (UID: \"a0aca2db-7f85-4df0-8440-ad3e9767d3c9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" Sep 07 13:02:41.676756 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.676569 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fjdp\" (UniqueName: \"kubernetes.io/projected/a0aca2db-7f85-4df0-8440-ad3e9767d3c9-kube-api-access-4fjdp\") pod \"prometheus-operator-6f4946db56-lj5t6\" (UID: \"a0aca2db-7f85-4df0-8440-ad3e9767d3c9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" Sep 07 13:02:41.676756 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.676608 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0aca2db-7f85-4df0-8440-ad3e9767d3c9-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-lj5t6\" (UID: \"a0aca2db-7f85-4df0-8440-ad3e9767d3c9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" Sep 07 13:02:41.676877 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:41.676841 2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 07 13:02:41.676943 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:41.676926 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0aca2db-7f85-4df0-8440-ad3e9767d3c9-prometheus-operator-tls podName:a0aca2db-7f85-4df0-8440-ad3e9767d3c9 nodeName:}" failed. No retries permitted until 2026-09-07 13:02:42.176903048 +0000 UTC m=+122.772407661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/a0aca2db-7f85-4df0-8440-ad3e9767d3c9-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-lj5t6" (UID: "a0aca2db-7f85-4df0-8440-ad3e9767d3c9") : secret "prometheus-operator-tls" not found Sep 07 13:02:41.677332 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.677313 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0aca2db-7f85-4df0-8440-ad3e9767d3c9-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-lj5t6\" (UID: \"a0aca2db-7f85-4df0-8440-ad3e9767d3c9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" Sep 07 13:02:41.679024 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.679002 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0aca2db-7f85-4df0-8440-ad3e9767d3c9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-lj5t6\" (UID: \"a0aca2db-7f85-4df0-8440-ad3e9767d3c9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" Sep 07 13:02:41.686879 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:41.686858 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fjdp\" (UniqueName: \"kubernetes.io/projected/a0aca2db-7f85-4df0-8440-ad3e9767d3c9-kube-api-access-4fjdp\") pod \"prometheus-operator-6f4946db56-lj5t6\" (UID: \"a0aca2db-7f85-4df0-8440-ad3e9767d3c9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" Sep 07 13:02:42.182689 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:42.182656 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a0aca2db-7f85-4df0-8440-ad3e9767d3c9-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-lj5t6\" (UID: \"a0aca2db-7f85-4df0-8440-ad3e9767d3c9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" Sep 07 13:02:42.184985 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:42.184951 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a0aca2db-7f85-4df0-8440-ad3e9767d3c9-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-lj5t6\" (UID: \"a0aca2db-7f85-4df0-8440-ad3e9767d3c9\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" Sep 07 13:02:42.362414 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:42.362323 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mkg7m" event={"ID":"845ff0c5-eab8-4c81-abca-19f574c1dcdb","Type":"ContainerStarted","Data":"0693f131a0b16ff358818b8f20ebbe663c162701070e363915442042fd3e0d1d"} Sep 07 13:02:42.381705 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:42.381659 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mkg7m" podStartSLOduration=33.72471536 podStartE2EDuration="36.381645124s" totalImagesPullingTime="2.656929764s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 13:02:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:02:42.380556215 +0000 UTC m=+122.976060830" watchObservedRunningTime="2026-09-07 13:02:42.381645124 +0000 UTC m=+122.977149741" Sep 07 13:02:42.421563 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:42.421527 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" Sep 07 13:02:42.691544 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:42.691471 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-lj5t6"] Sep 07 13:02:42.694362 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:02:42.694336 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0aca2db_7f85_4df0_8440_ad3e9767d3c9.slice/crio-b2aa61eaebd4749f03e60ecfbee2e0141f6073c144150b40f6351f4164d5ffb3 WatchSource:0}: Error finding container b2aa61eaebd4749f03e60ecfbee2e0141f6073c144150b40f6351f4164d5ffb3: Status 404 returned error can't find the container with id b2aa61eaebd4749f03e60ecfbee2e0141f6073c144150b40f6351f4164d5ffb3 Sep 07 13:02:43.366856 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:43.366817 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" event={"ID":"a0aca2db-7f85-4df0-8440-ad3e9767d3c9","Type":"ContainerStarted","Data":"b2aa61eaebd4749f03e60ecfbee2e0141f6073c144150b40f6351f4164d5ffb3"} Sep 07 13:02:44.371706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:44.371615 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" event={"ID":"a0aca2db-7f85-4df0-8440-ad3e9767d3c9","Type":"ContainerStarted","Data":"143175b180a3a01fe1173e5fc41dd8aacd9f75ecd38a40df001caa6c8a0ea95b"} Sep 07 13:02:44.371706 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:44.371682 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" event={"ID":"a0aca2db-7f85-4df0-8440-ad3e9767d3c9","Type":"ContainerStarted","Data":"a78b8eafc1f487b8180b4e2ae1a141696293daab431c806b09f529e32cf2d09a"} Sep 07 13:02:44.395764 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:44.395704 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-lj5t6" podStartSLOduration=2.024284239 podStartE2EDuration="3.395689566s" totalImagesPullingTime="1.371405327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:02:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:02:44.395108111 +0000 UTC m=+124.990612728" watchObservedRunningTime="2026-09-07 13:02:44.395689566 +0000 UTC m=+124.991194181" Sep 07 13:02:45.876999 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:45.876964 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr"] Sep 07 13:02:45.880504 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:45.880486 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" Sep 07 13:02:45.883560 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:45.883532 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 07 13:02:45.883700 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:45.883599 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 07 13:02:45.883767 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:45.883721 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-lqt5j\"" Sep 07 13:02:45.892293 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:45.892259 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr"] Sep 07 13:02:45.915977 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:45.915946 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce4d061f-a135-4f03-8326-8eccb67ce345-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-frjrr\" (UID: \"ce4d061f-a135-4f03-8326-8eccb67ce345\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" Sep 07 13:02:45.916133 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:45.915987 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-526rz\" (UniqueName: \"kubernetes.io/projected/ce4d061f-a135-4f03-8326-8eccb67ce345-kube-api-access-526rz\") pod \"openshift-state-metrics-657fd6c54-frjrr\" (UID: \"ce4d061f-a135-4f03-8326-8eccb67ce345\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" Sep 07 13:02:45.916133 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:45.916015 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce4d061f-a135-4f03-8326-8eccb67ce345-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-frjrr\" (UID: \"ce4d061f-a135-4f03-8326-8eccb67ce345\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" Sep 07 13:02:45.916133 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:45.916098 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce4d061f-a135-4f03-8326-8eccb67ce345-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-frjrr\" (UID: \"ce4d061f-a135-4f03-8326-8eccb67ce345\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" Sep 07 13:02:45.922458 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:45.922434 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7gqks"] Sep 07 13:02:45.925397 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:45.925380 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:45.927650 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:45.927625 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 07 13:02:45.927816 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:45.927790 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 07 13:02:45.927887 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:45.927852 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hgwh9\"" Sep 07 13:02:45.928024 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:45.927989 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 07 13:02:46.017325 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.017292 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/902a96ad-535c-44a0-b497-56b5f03ff2aa-sys\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.017325 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.017325 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/902a96ad-535c-44a0-b497-56b5f03ff2aa-node-exporter-textfile\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.017538 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.017343 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/902a96ad-535c-44a0-b497-56b5f03ff2aa-node-exporter-wtmp\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.017538 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.017363 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce4d061f-a135-4f03-8326-8eccb67ce345-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-frjrr\" (UID: \"ce4d061f-a135-4f03-8326-8eccb67ce345\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" Sep 07 13:02:46.017538 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.017393 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mk4c\" (UniqueName: \"kubernetes.io/projected/902a96ad-535c-44a0-b497-56b5f03ff2aa-kube-api-access-8mk4c\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.017538 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.017408 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/902a96ad-535c-44a0-b497-56b5f03ff2aa-root\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.017538 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.017428 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/902a96ad-535c-44a0-b497-56b5f03ff2aa-metrics-client-ca\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.017538 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.017444 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/902a96ad-535c-44a0-b497-56b5f03ff2aa-node-exporter-accelerators-collector-config\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.017538 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.017506 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/902a96ad-535c-44a0-b497-56b5f03ff2aa-node-exporter-tls\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.017774 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.017562 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce4d061f-a135-4f03-8326-8eccb67ce345-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-frjrr\" (UID: \"ce4d061f-a135-4f03-8326-8eccb67ce345\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" Sep 07 13:02:46.017774 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.017579 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/902a96ad-535c-44a0-b497-56b5f03ff2aa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.017774 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.017621 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-526rz\" (UniqueName: \"kubernetes.io/projected/ce4d061f-a135-4f03-8326-8eccb67ce345-kube-api-access-526rz\") pod \"openshift-state-metrics-657fd6c54-frjrr\" (UID: \"ce4d061f-a135-4f03-8326-8eccb67ce345\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" Sep 07 13:02:46.017774 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.017691 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce4d061f-a135-4f03-8326-8eccb67ce345-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-frjrr\" (UID: \"ce4d061f-a135-4f03-8326-8eccb67ce345\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" Sep 07 13:02:46.017774 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:46.017717 2564 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 07 13:02:46.017945 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:46.017778 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce4d061f-a135-4f03-8326-8eccb67ce345-openshift-state-metrics-tls podName:ce4d061f-a135-4f03-8326-8eccb67ce345 nodeName:}" failed. No retries permitted until 2026-09-07 13:02:46.517760421 +0000 UTC m=+127.113265014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ce4d061f-a135-4f03-8326-8eccb67ce345-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-frjrr" (UID: "ce4d061f-a135-4f03-8326-8eccb67ce345") : secret "openshift-state-metrics-tls" not found Sep 07 13:02:46.017988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.017978 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce4d061f-a135-4f03-8326-8eccb67ce345-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-frjrr\" (UID: \"ce4d061f-a135-4f03-8326-8eccb67ce345\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" Sep 07 13:02:46.019925 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.019907 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce4d061f-a135-4f03-8326-8eccb67ce345-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-frjrr\" (UID: \"ce4d061f-a135-4f03-8326-8eccb67ce345\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" Sep 07 13:02:46.027399 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.027375 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-526rz\" (UniqueName: \"kubernetes.io/projected/ce4d061f-a135-4f03-8326-8eccb67ce345-kube-api-access-526rz\") pod \"openshift-state-metrics-657fd6c54-frjrr\" (UID: \"ce4d061f-a135-4f03-8326-8eccb67ce345\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" Sep 07 13:02:46.118674 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.118639 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/902a96ad-535c-44a0-b497-56b5f03ff2aa-sys\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.118674 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.118679 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/902a96ad-535c-44a0-b497-56b5f03ff2aa-node-exporter-textfile\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.118878 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.118699 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/902a96ad-535c-44a0-b497-56b5f03ff2aa-node-exporter-wtmp\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.118878 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.118731 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mk4c\" (UniqueName: \"kubernetes.io/projected/902a96ad-535c-44a0-b497-56b5f03ff2aa-kube-api-access-8mk4c\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.118878 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.118750 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/902a96ad-535c-44a0-b497-56b5f03ff2aa-root\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.118878 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.118749 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/902a96ad-535c-44a0-b497-56b5f03ff2aa-sys\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.118878 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.118772 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/902a96ad-535c-44a0-b497-56b5f03ff2aa-metrics-client-ca\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.118878 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.118811 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/902a96ad-535c-44a0-b497-56b5f03ff2aa-root\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.118878 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.118811 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/902a96ad-535c-44a0-b497-56b5f03ff2aa-node-exporter-accelerators-collector-config\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.118878 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.118848 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/902a96ad-535c-44a0-b497-56b5f03ff2aa-node-exporter-tls\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.119290 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.118905 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/902a96ad-535c-44a0-b497-56b5f03ff2aa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.119290 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.118908 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/902a96ad-535c-44a0-b497-56b5f03ff2aa-node-exporter-wtmp\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.119290 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.119014 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/902a96ad-535c-44a0-b497-56b5f03ff2aa-node-exporter-textfile\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.119437 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.119292 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/902a96ad-535c-44a0-b497-56b5f03ff2aa-node-exporter-accelerators-collector-config\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.119495 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.119461 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/902a96ad-535c-44a0-b497-56b5f03ff2aa-metrics-client-ca\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.121150 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.121130 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/902a96ad-535c-44a0-b497-56b5f03ff2aa-node-exporter-tls\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.121286 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.121269 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/902a96ad-535c-44a0-b497-56b5f03ff2aa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.127774 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.127725 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mk4c\" (UniqueName: \"kubernetes.io/projected/902a96ad-535c-44a0-b497-56b5f03ff2aa-kube-api-access-8mk4c\") pod \"node-exporter-7gqks\" (UID: \"902a96ad-535c-44a0-b497-56b5f03ff2aa\") " pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.234240 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.234198 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7gqks" Sep 07 13:02:46.245685 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:02:46.245656 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod902a96ad_535c_44a0_b497_56b5f03ff2aa.slice/crio-7a1d988ad910d6f9c899b6112dc33126f50126a5b9693d64f4a6e7675ac0c554 WatchSource:0}: Error finding container 7a1d988ad910d6f9c899b6112dc33126f50126a5b9693d64f4a6e7675ac0c554: Status 404 returned error can't find the container with id 7a1d988ad910d6f9c899b6112dc33126f50126a5b9693d64f4a6e7675ac0c554 Sep 07 13:02:46.382140 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.382060 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7gqks" event={"ID":"902a96ad-535c-44a0-b497-56b5f03ff2aa","Type":"ContainerStarted","Data":"7a1d988ad910d6f9c899b6112dc33126f50126a5b9693d64f4a6e7675ac0c554"} Sep 07 13:02:46.523593 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.523558 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce4d061f-a135-4f03-8326-8eccb67ce345-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-frjrr\" (UID: \"ce4d061f-a135-4f03-8326-8eccb67ce345\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" Sep 07 13:02:46.525846 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.525826 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce4d061f-a135-4f03-8326-8eccb67ce345-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-frjrr\" (UID: \"ce4d061f-a135-4f03-8326-8eccb67ce345\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" Sep 07 13:02:46.789276 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.789238 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" Sep 07 13:02:46.997070 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:46.996989 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 13:02:47.000916 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.000890 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.004371 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.004346 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 07 13:02:47.004932 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.004750 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 07 13:02:47.004932 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.004758 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 07 13:02:47.004932 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.004758 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 07 13:02:47.004932 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.004819 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 07 13:02:47.005237 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.005218 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9kr8m\"" Sep 07 13:02:47.005300 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.005218 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 07 13:02:47.005300 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.005252 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 07 13:02:47.005497 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.005479 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 07 13:02:47.005601 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.005584 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 07 13:02:47.021148 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.021124 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 13:02:47.086843 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.086765 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr"] Sep 07 13:02:47.128406 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.128372 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd8sd\" (UniqueName: \"kubernetes.io/projected/6d0cb95c-fbda-4e94-8d16-46a6c197b822-kube-api-access-xd8sd\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.128574 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.128417 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-config-volume\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.128574 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.128470 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.128574 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.128504 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.128574 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.128529 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6d0cb95c-fbda-4e94-8d16-46a6c197b822-config-out\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.128574 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.128568 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6d0cb95c-fbda-4e94-8d16-46a6c197b822-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.128774 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.128598 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.128774 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.128661 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d0cb95c-fbda-4e94-8d16-46a6c197b822-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.128774 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.128682 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d0cb95c-fbda-4e94-8d16-46a6c197b822-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.128774 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.128731 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.128934 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.128779 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-web-config\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.128934 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.128817 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.128934 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.128865 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6d0cb95c-fbda-4e94-8d16-46a6c197b822-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.229414 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.229377 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd8sd\" (UniqueName: \"kubernetes.io/projected/6d0cb95c-fbda-4e94-8d16-46a6c197b822-kube-api-access-xd8sd\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.229414 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.229417 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-config-volume\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.229651 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.229457 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.229651 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.229504 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.229651 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.229543 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6d0cb95c-fbda-4e94-8d16-46a6c197b822-config-out\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.229805 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.229648 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6d0cb95c-fbda-4e94-8d16-46a6c197b822-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.229805 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.229682 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.229805 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.229731 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d0cb95c-fbda-4e94-8d16-46a6c197b822-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.229805 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.229755 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d0cb95c-fbda-4e94-8d16-46a6c197b822-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.230008 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.229808 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.230008 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.229836 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-web-config\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.230008 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.229863 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.230008 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.229899 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6d0cb95c-fbda-4e94-8d16-46a6c197b822-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.230008 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:47.229965 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d0cb95c-fbda-4e94-8d16-46a6c197b822-alertmanager-trusted-ca-bundle podName:6d0cb95c-fbda-4e94-8d16-46a6c197b822 nodeName:}" failed. No retries permitted until 2026-09-07 13:02:47.729944159 +0000 UTC m=+128.325448765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/6d0cb95c-fbda-4e94-8d16-46a6c197b822-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "6d0cb95c-fbda-4e94-8d16-46a6c197b822") : configmap references non-existent config key: ca-bundle.crt Sep 07 13:02:47.231457 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.230871 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d0cb95c-fbda-4e94-8d16-46a6c197b822-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.232040 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.231886 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6d0cb95c-fbda-4e94-8d16-46a6c197b822-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.232741 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.232693 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-config-volume\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.232821 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.232797 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.233385 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.233356 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.234004 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.233960 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.234292 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.234251 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6d0cb95c-fbda-4e94-8d16-46a6c197b822-config-out\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.235277 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.235245 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-web-config\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.235451 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.235424 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.235544 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.235486 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.235688 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.235654 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6d0cb95c-fbda-4e94-8d16-46a6c197b822-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.248930 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.248900 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd8sd\" (UniqueName: \"kubernetes.io/projected/6d0cb95c-fbda-4e94-8d16-46a6c197b822-kube-api-access-xd8sd\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.306401 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:02:47.306370 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce4d061f_a135_4f03_8326_8eccb67ce345.slice/crio-b74c7fae7e3b3c8b6bc98b5d11af16d5b38f39e79d5a6b3e45ee9b13ac8d648c WatchSource:0}: Error finding container b74c7fae7e3b3c8b6bc98b5d11af16d5b38f39e79d5a6b3e45ee9b13ac8d648c: Status 404 returned error can't find the container with id b74c7fae7e3b3c8b6bc98b5d11af16d5b38f39e79d5a6b3e45ee9b13ac8d648c Sep 07 13:02:47.397703 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.397665 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" event={"ID":"ce4d061f-a135-4f03-8326-8eccb67ce345","Type":"ContainerStarted","Data":"82d88e8ad380bb74d503ebffc5e3e0a63662cb32d15f676fc510578552e071f9"} Sep 07 13:02:47.397825 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.397777 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" event={"ID":"ce4d061f-a135-4f03-8326-8eccb67ce345","Type":"ContainerStarted","Data":"b74c7fae7e3b3c8b6bc98b5d11af16d5b38f39e79d5a6b3e45ee9b13ac8d648c"} Sep 07 13:02:47.582493 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.582452 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-zzch6"] Sep 07 13:02:47.585584 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.585562 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-zzch6" Sep 07 13:02:47.589002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.588978 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-hd4n7\"" Sep 07 13:02:47.589833 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.589809 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 07 13:02:47.589914 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.589881 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 07 13:02:47.602286 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.602251 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-zzch6"] Sep 07 13:02:47.633409 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.633378 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zww8p\" (UniqueName: \"kubernetes.io/projected/2458852c-2282-4634-9422-ff5843ccfb68-kube-api-access-zww8p\") pod \"downloads-85d98c998f-zzch6\" (UID: \"2458852c-2282-4634-9422-ff5843ccfb68\") " pod="openshift-console/downloads-85d98c998f-zzch6" Sep 07 13:02:47.733802 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.733768 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d0cb95c-fbda-4e94-8d16-46a6c197b822-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.733974 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.733841 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zww8p\" (UniqueName: \"kubernetes.io/projected/2458852c-2282-4634-9422-ff5843ccfb68-kube-api-access-zww8p\") pod \"downloads-85d98c998f-zzch6\" (UID: \"2458852c-2282-4634-9422-ff5843ccfb68\") " pod="openshift-console/downloads-85d98c998f-zzch6" Sep 07 13:02:47.734817 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.734788 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d0cb95c-fbda-4e94-8d16-46a6c197b822-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:47.742294 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.742271 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zww8p\" (UniqueName: \"kubernetes.io/projected/2458852c-2282-4634-9422-ff5843ccfb68-kube-api-access-zww8p\") pod \"downloads-85d98c998f-zzch6\" (UID: \"2458852c-2282-4634-9422-ff5843ccfb68\") " pod="openshift-console/downloads-85d98c998f-zzch6" Sep 07 13:02:47.895028 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.894942 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-zzch6" Sep 07 13:02:47.913238 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:47.913208 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:02:48.191304 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:48.191182 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-zzch6"] Sep 07 13:02:48.204067 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:02:48.204018 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2458852c_2282_4634_9422_ff5843ccfb68.slice/crio-2dd52715cf8eed2a1b6e2982cb0ce1aa1b0314f60e94afb50e46e212612625fa WatchSource:0}: Error finding container 2dd52715cf8eed2a1b6e2982cb0ce1aa1b0314f60e94afb50e46e212612625fa: Status 404 returned error can't find the container with id 2dd52715cf8eed2a1b6e2982cb0ce1aa1b0314f60e94afb50e46e212612625fa Sep 07 13:02:48.237082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:48.237040 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 13:02:48.239245 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:02:48.239218 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d0cb95c_fbda_4e94_8d16_46a6c197b822.slice/crio-cb4ee053a7d794577aa2493cdf39fde52642bf74920e6e3d9414025d485ba1db WatchSource:0}: Error finding container cb4ee053a7d794577aa2493cdf39fde52642bf74920e6e3d9414025d485ba1db: Status 404 returned error can't find the container with id cb4ee053a7d794577aa2493cdf39fde52642bf74920e6e3d9414025d485ba1db Sep 07 13:02:48.402866 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:48.402832 2564 generic.go:356] "Generic (PLEG): container finished" podID="902a96ad-535c-44a0-b497-56b5f03ff2aa" containerID="e0fab778ed284fd223254d34789691d4c6e1248fb7168fb29fbe8427643a63f0" exitCode=0 Sep 07 13:02:48.403034 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:48.402913 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7gqks" event={"ID":"902a96ad-535c-44a0-b497-56b5f03ff2aa","Type":"ContainerDied","Data":"e0fab778ed284fd223254d34789691d4c6e1248fb7168fb29fbe8427643a63f0"} Sep 07 13:02:48.404294 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:48.404269 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d0cb95c-fbda-4e94-8d16-46a6c197b822","Type":"ContainerStarted","Data":"cb4ee053a7d794577aa2493cdf39fde52642bf74920e6e3d9414025d485ba1db"} Sep 07 13:02:48.405458 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:48.405427 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-zzch6" event={"ID":"2458852c-2282-4634-9422-ff5843ccfb68","Type":"ContainerStarted","Data":"2dd52715cf8eed2a1b6e2982cb0ce1aa1b0314f60e94afb50e46e212612625fa"} Sep 07 13:02:48.407460 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:48.407433 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" event={"ID":"ce4d061f-a135-4f03-8326-8eccb67ce345","Type":"ContainerStarted","Data":"da18e1290649987fea676d608bb8542be871e5ecfbf85dec24a6e5910746e909"} Sep 07 13:02:49.415260 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:49.415222 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7gqks" event={"ID":"902a96ad-535c-44a0-b497-56b5f03ff2aa","Type":"ContainerStarted","Data":"10ec195f9e040cde8bfeb38c4a8a6babd6e87d3eb6efa7f701b93bbadb8c1a5d"} Sep 07 13:02:49.415775 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:49.415286 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7gqks" event={"ID":"902a96ad-535c-44a0-b497-56b5f03ff2aa","Type":"ContainerStarted","Data":"2e9b00c991f82031f9a6d5d7bc0b3f8187c715c32d11127822f6e1253d751f3b"} Sep 07 13:02:49.417414 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:49.417383 2564 generic.go:356] "Generic (PLEG): container finished" podID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerID="5cce426dbc9feb788549b269b1f38e2cce9f35d5fc62cccf378ec540a1eca267" exitCode=0 Sep 07 13:02:49.417578 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:49.417484 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d0cb95c-fbda-4e94-8d16-46a6c197b822","Type":"ContainerDied","Data":"5cce426dbc9feb788549b269b1f38e2cce9f35d5fc62cccf378ec540a1eca267"} Sep 07 13:02:49.420852 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:49.420821 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" event={"ID":"ce4d061f-a135-4f03-8326-8eccb67ce345","Type":"ContainerStarted","Data":"948c3d614fdb6b54c5e6812510a4a7e1bade5dc43f3da0455ab69e8b158236b8"} Sep 07 13:02:49.442652 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:49.442588 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7gqks" podStartSLOduration=3.273600303 podStartE2EDuration="4.442570457s" totalImagesPullingTime="1.103422851s" totalInitContainerRuntime="65.547303ms" isStatefulPod=true podCreationTimestamp="2026-09-07 13:02:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:02:49.441995668 +0000 UTC m=+130.037500290" watchObservedRunningTime="2026-09-07 13:02:49.442570457 +0000 UTC m=+130.038075156" Sep 07 13:02:49.497547 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:49.497478 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-frjrr" podStartSLOduration=2.9556579579999998 podStartE2EDuration="4.49745545s" totalImagesPullingTime="1.541797492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:02:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:02:49.495890671 +0000 UTC m=+130.091395287" watchObservedRunningTime="2026-09-07 13:02:49.49745545 +0000 UTC m=+130.092960067" Sep 07 13:02:49.651475 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:49.651387 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs\") pod \"network-metrics-daemon-f5cpd\" (UID: \"5fbd07ff-dbb9-46f0-904d-d21419e31373\") " pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:02:49.654141 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:49.654111 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fbd07ff-dbb9-46f0-904d-d21419e31373-metrics-certs\") pod \"network-metrics-daemon-f5cpd\" (UID: \"5fbd07ff-dbb9-46f0-904d-d21419e31373\") " pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:02:49.738126 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:49.738086 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sgcc8\"" Sep 07 13:02:49.745512 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:49.745480 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5cpd" Sep 07 13:02:50.025087 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:50.025039 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f5cpd"] Sep 07 13:02:50.027935 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:02:50.027896 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fbd07ff_dbb9_46f0_904d_d21419e31373.slice/crio-ee93f76a15e32d0f643242824050661e1b82f839d719163d3fc0c41f8769476e WatchSource:0}: Error finding container ee93f76a15e32d0f643242824050661e1b82f839d719163d3fc0c41f8769476e: Status 404 returned error can't find the container with id ee93f76a15e32d0f643242824050661e1b82f839d719163d3fc0c41f8769476e Sep 07 13:02:50.427401 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:50.427310 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f5cpd" event={"ID":"5fbd07ff-dbb9-46f0-904d-d21419e31373","Type":"ContainerStarted","Data":"ee93f76a15e32d0f643242824050661e1b82f839d719163d3fc0c41f8769476e"} Sep 07 13:02:50.629539 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:50.628116 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq"] Sep 07 13:02:50.632324 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:50.632296 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq" Sep 07 13:02:50.635001 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:50.634982 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 07 13:02:50.635207 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:50.634993 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-n56wc\"" Sep 07 13:02:50.640606 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:50.640579 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq"] Sep 07 13:02:50.761537 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:50.761490 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br6cr\" (UniqueName: \"kubernetes.io/projected/723b3e1d-a3f4-491f-b45e-f20b1972b28c-kube-api-access-br6cr\") pod \"monitoring-plugin-85bbbbbf4-th7sq\" (UID: \"723b3e1d-a3f4-491f-b45e-f20b1972b28c\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq" Sep 07 13:02:50.761717 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:50.761589 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/723b3e1d-a3f4-491f-b45e-f20b1972b28c-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-th7sq\" (UID: \"723b3e1d-a3f4-491f-b45e-f20b1972b28c\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq" Sep 07 13:02:50.862582 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:50.862533 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/723b3e1d-a3f4-491f-b45e-f20b1972b28c-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-th7sq\" (UID: \"723b3e1d-a3f4-491f-b45e-f20b1972b28c\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq" Sep 07 13:02:50.862774 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:50.862621 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br6cr\" (UniqueName: \"kubernetes.io/projected/723b3e1d-a3f4-491f-b45e-f20b1972b28c-kube-api-access-br6cr\") pod \"monitoring-plugin-85bbbbbf4-th7sq\" (UID: \"723b3e1d-a3f4-491f-b45e-f20b1972b28c\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq" Sep 07 13:02:50.862774 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:50.862705 2564 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 07 13:02:50.862882 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:02:50.862792 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/723b3e1d-a3f4-491f-b45e-f20b1972b28c-monitoring-plugin-cert podName:723b3e1d-a3f4-491f-b45e-f20b1972b28c nodeName:}" failed. No retries permitted until 2026-09-07 13:02:51.362769879 +0000 UTC m=+131.958274490 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/723b3e1d-a3f4-491f-b45e-f20b1972b28c-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-th7sq" (UID: "723b3e1d-a3f4-491f-b45e-f20b1972b28c") : secret "monitoring-plugin-cert" not found Sep 07 13:02:50.878101 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:50.878068 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br6cr\" (UniqueName: \"kubernetes.io/projected/723b3e1d-a3f4-491f-b45e-f20b1972b28c-kube-api-access-br6cr\") pod \"monitoring-plugin-85bbbbbf4-th7sq\" (UID: \"723b3e1d-a3f4-491f-b45e-f20b1972b28c\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq" Sep 07 13:02:51.367313 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:51.367276 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/723b3e1d-a3f4-491f-b45e-f20b1972b28c-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-th7sq\" (UID: \"723b3e1d-a3f4-491f-b45e-f20b1972b28c\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq" Sep 07 13:02:51.369929 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:51.369899 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/723b3e1d-a3f4-491f-b45e-f20b1972b28c-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-th7sq\" (UID: \"723b3e1d-a3f4-491f-b45e-f20b1972b28c\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq" Sep 07 13:02:51.546220 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:51.546191 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq" Sep 07 13:02:51.847149 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:51.847114 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq"] Sep 07 13:02:51.850436 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:02:51.850404 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod723b3e1d_a3f4_491f_b45e_f20b1972b28c.slice/crio-539f8a0167b79e91ac996fae4a182029f794419b0aee57e4cb2631f2a62deb3e WatchSource:0}: Error finding container 539f8a0167b79e91ac996fae4a182029f794419b0aee57e4cb2631f2a62deb3e: Status 404 returned error can't find the container with id 539f8a0167b79e91ac996fae4a182029f794419b0aee57e4cb2631f2a62deb3e Sep 07 13:02:52.441575 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:52.441532 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f5cpd" event={"ID":"5fbd07ff-dbb9-46f0-904d-d21419e31373","Type":"ContainerStarted","Data":"c64d8bf4808b665b676614f4e4fac5878fc83de0fd9687aae6ae4d3114cae9b1"} Sep 07 13:02:52.441757 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:52.441602 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f5cpd" event={"ID":"5fbd07ff-dbb9-46f0-904d-d21419e31373","Type":"ContainerStarted","Data":"4e9a181472fcdb4fc4b0cd55d22a5eac846ff6fc94bd7ff5dc779e5ceae06704"} Sep 07 13:02:52.443426 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:52.443393 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq" event={"ID":"723b3e1d-a3f4-491f-b45e-f20b1972b28c","Type":"ContainerStarted","Data":"539f8a0167b79e91ac996fae4a182029f794419b0aee57e4cb2631f2a62deb3e"} Sep 07 13:02:52.450824 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:52.450716 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d0cb95c-fbda-4e94-8d16-46a6c197b822","Type":"ContainerStarted","Data":"bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0"} Sep 07 13:02:52.450948 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:52.450903 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d0cb95c-fbda-4e94-8d16-46a6c197b822","Type":"ContainerStarted","Data":"b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626"} Sep 07 13:02:52.451005 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:52.450953 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d0cb95c-fbda-4e94-8d16-46a6c197b822","Type":"ContainerStarted","Data":"caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a"} Sep 07 13:02:52.451005 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:52.450996 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d0cb95c-fbda-4e94-8d16-46a6c197b822","Type":"ContainerStarted","Data":"8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357"} Sep 07 13:02:52.451120 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:52.451036 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d0cb95c-fbda-4e94-8d16-46a6c197b822","Type":"ContainerStarted","Data":"40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c"} Sep 07 13:02:52.471279 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:52.471231 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-f5cpd" podStartSLOduration=131.069256402 podStartE2EDuration="2m12.471211659s" totalImagesPullingTime="1.401955257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:02:52.469879542 +0000 UTC m=+133.065384159" watchObservedRunningTime="2026-09-07 13:02:52.471211659 +0000 UTC m=+133.066716275" Sep 07 13:02:53.460304 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:53.460265 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d0cb95c-fbda-4e94-8d16-46a6c197b822","Type":"ContainerStarted","Data":"f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc"} Sep 07 13:02:53.491569 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:53.491503 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.632704818 podStartE2EDuration="7.491484494s" totalImagesPullingTime="3.842617684s" totalInitContainerRuntime="16.161992ms" isStatefulPod=false podCreationTimestamp="2026-09-07 13:02:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:02:53.489455159 +0000 UTC m=+134.084959799" watchObservedRunningTime="2026-09-07 13:02:53.491484494 +0000 UTC m=+134.086989155" Sep 07 13:02:54.468035 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:54.467982 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq" event={"ID":"723b3e1d-a3f4-491f-b45e-f20b1972b28c","Type":"ContainerStarted","Data":"7cc4bd52a67c9d7fe60609f5d8c4259d308b04db229a55128b7979b79b038660"} Sep 07 13:02:54.468504 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:54.468477 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq" Sep 07 13:02:54.474283 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:54.474256 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq" Sep 07 13:02:54.490589 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:54.490467 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-th7sq" podStartSLOduration=2.913318204 podStartE2EDuration="4.490445153s" totalImagesPullingTime="1.577126949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:02:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:02:54.490035643 +0000 UTC m=+135.085540286" watchObservedRunningTime="2026-09-07 13:02:54.490445153 +0000 UTC m=+135.085949769" Sep 07 13:02:56.794093 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:56.794039 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-864459d56-bbg5b"] Sep 07 13:02:56.854341 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:56.854306 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76f6584dc7-spwmw"] Sep 07 13:02:56.858108 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:56.858082 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:56.860926 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:56.860903 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 07 13:02:56.861322 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:56.861300 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 07 13:02:56.861420 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:56.861309 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 07 13:02:56.861705 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:56.861684 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 07 13:02:56.862248 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:56.861998 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jwmrq\"" Sep 07 13:02:56.862248 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:56.862027 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 07 13:02:56.874540 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:56.874514 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76f6584dc7-spwmw"] Sep 07 13:02:56.922809 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:56.922767 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj7vj\" (UniqueName: \"kubernetes.io/projected/d367bdb3-534a-4216-8fc7-16f2dee0d838-kube-api-access-jj7vj\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:56.922990 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:56.922817 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-service-ca\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:56.922990 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:56.922881 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-serving-cert\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:56.922990 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:56.922931 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-config\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:56.923181 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:56.923036 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-oauth-serving-cert\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:56.923181 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:56.923116 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-oauth-config\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:57.024305 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:57.024267 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-oauth-serving-cert\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:57.024483 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:57.024331 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-oauth-config\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:57.024483 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:57.024387 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj7vj\" (UniqueName: \"kubernetes.io/projected/d367bdb3-534a-4216-8fc7-16f2dee0d838-kube-api-access-jj7vj\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:57.024483 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:57.024416 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-service-ca\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:57.024483 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:57.024447 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-serving-cert\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:57.024713 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:57.024560 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-config\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:57.025218 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:57.025166 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-oauth-serving-cert\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:57.025419 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:57.025395 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-service-ca\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:57.025508 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:57.025403 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-config\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:57.027181 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:57.027152 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-serving-cert\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:57.027373 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:57.027349 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-oauth-config\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:57.040065 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:57.040022 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj7vj\" (UniqueName: \"kubernetes.io/projected/d367bdb3-534a-4216-8fc7-16f2dee0d838-kube-api-access-jj7vj\") pod \"console-76f6584dc7-spwmw\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:57.168757 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:57.168640 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:02:57.461938 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:57.461852 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76f6584dc7-spwmw"] Sep 07 13:02:57.466029 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:02:57.465994 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd367bdb3_534a_4216_8fc7_16f2dee0d838.slice/crio-3d6e2d867341a38742ac827863a4345ff34fa2861dfe63b7883a5a12399980da WatchSource:0}: Error finding container 3d6e2d867341a38742ac827863a4345ff34fa2861dfe63b7883a5a12399980da: Status 404 returned error can't find the container with id 3d6e2d867341a38742ac827863a4345ff34fa2861dfe63b7883a5a12399980da Sep 07 13:02:57.484532 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:02:57.484495 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76f6584dc7-spwmw" event={"ID":"d367bdb3-534a-4216-8fc7-16f2dee0d838","Type":"ContainerStarted","Data":"3d6e2d867341a38742ac827863a4345ff34fa2861dfe63b7883a5a12399980da"} Sep 07 13:03:07.536583 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:07.536537 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-zzch6" event={"ID":"2458852c-2282-4634-9422-ff5843ccfb68","Type":"ContainerStarted","Data":"8e9808ea8e81666842a5d35a783484b77fcc7a76e8e99c2c957594d6407b674a"} Sep 07 13:03:07.537131 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:07.536826 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-zzch6" Sep 07 13:03:07.538845 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:07.538640 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76f6584dc7-spwmw" event={"ID":"d367bdb3-534a-4216-8fc7-16f2dee0d838","Type":"ContainerStarted","Data":"49217c7522e2da957c0491229ab6c71ef634727692707620e01df12f208fa16e"} Sep 07 13:03:07.556206 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:07.556175 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-zzch6" Sep 07 13:03:07.557618 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:07.557565 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-zzch6" podStartSLOduration=1.628069776 podStartE2EDuration="20.55754728s" totalImagesPullingTime="18.929477504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:02:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:03:07.556540857 +0000 UTC m=+148.152045473" watchObservedRunningTime="2026-09-07 13:03:07.55754728 +0000 UTC m=+148.153051897" Sep 07 13:03:07.597639 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:07.597582 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76f6584dc7-spwmw" podStartSLOduration=1.9683791529999999 podStartE2EDuration="11.597562783s" totalImagesPullingTime="9.62918363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:02:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:03:07.59586262 +0000 UTC m=+148.191367249" watchObservedRunningTime="2026-09-07 13:03:07.597562783 +0000 UTC m=+148.193067399" Sep 07 13:03:15.337041 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:15.337007 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76f6584dc7-spwmw"] Sep 07 13:03:17.168966 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:17.168920 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:03:21.816526 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:21.816481 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-864459d56-bbg5b" podUID="4b87e5f6-90fc-4891-aa11-feb499ce5511" containerName="registry" containerID="cri-o://4ac9a49d7df276c2d12dd7d2c274a1dcbab89791c6147576d8dfbb18f2045cf1" gracePeriod=30 Sep 07 13:03:22.090048 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.089981 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:03:22.137363 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.137337 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b87e5f6-90fc-4891-aa11-feb499ce5511-trusted-ca\") pod \"4b87e5f6-90fc-4891-aa11-feb499ce5511\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " Sep 07 13:03:22.137525 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.137382 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b87e5f6-90fc-4891-aa11-feb499ce5511-ca-trust-extracted\") pod \"4b87e5f6-90fc-4891-aa11-feb499ce5511\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " Sep 07 13:03:22.137525 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.137407 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-certificates\") pod \"4b87e5f6-90fc-4891-aa11-feb499ce5511\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " Sep 07 13:03:22.137525 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.137445 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b87e5f6-90fc-4891-aa11-feb499ce5511-image-registry-private-configuration\") pod \"4b87e5f6-90fc-4891-aa11-feb499ce5511\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " Sep 07 13:03:22.137525 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.137496 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b87e5f6-90fc-4891-aa11-feb499ce5511-installation-pull-secrets\") pod \"4b87e5f6-90fc-4891-aa11-feb499ce5511\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " Sep 07 13:03:22.137525 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.137522 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-bound-sa-token\") pod \"4b87e5f6-90fc-4891-aa11-feb499ce5511\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " Sep 07 13:03:22.137828 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.137551 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls\") pod \"4b87e5f6-90fc-4891-aa11-feb499ce5511\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " Sep 07 13:03:22.137828 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.137604 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87r5w\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-kube-api-access-87r5w\") pod \"4b87e5f6-90fc-4891-aa11-feb499ce5511\" (UID: \"4b87e5f6-90fc-4891-aa11-feb499ce5511\") " Sep 07 13:03:22.137942 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.137826 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b87e5f6-90fc-4891-aa11-feb499ce5511-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4b87e5f6-90fc-4891-aa11-feb499ce5511" (UID: "4b87e5f6-90fc-4891-aa11-feb499ce5511"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:03:22.137942 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.137830 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4b87e5f6-90fc-4891-aa11-feb499ce5511" (UID: "4b87e5f6-90fc-4891-aa11-feb499ce5511"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:03:22.140149 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.140012 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4b87e5f6-90fc-4891-aa11-feb499ce5511" (UID: "4b87e5f6-90fc-4891-aa11-feb499ce5511"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:03:22.140149 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.140069 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b87e5f6-90fc-4891-aa11-feb499ce5511-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4b87e5f6-90fc-4891-aa11-feb499ce5511" (UID: "4b87e5f6-90fc-4891-aa11-feb499ce5511"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:03:22.140357 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.140203 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b87e5f6-90fc-4891-aa11-feb499ce5511-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4b87e5f6-90fc-4891-aa11-feb499ce5511" (UID: "4b87e5f6-90fc-4891-aa11-feb499ce5511"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:03:22.140357 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.140337 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-kube-api-access-87r5w" (OuterVolumeSpecName: "kube-api-access-87r5w") pod "4b87e5f6-90fc-4891-aa11-feb499ce5511" (UID: "4b87e5f6-90fc-4891-aa11-feb499ce5511"). InnerVolumeSpecName "kube-api-access-87r5w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:03:22.140438 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.140383 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4b87e5f6-90fc-4891-aa11-feb499ce5511" (UID: "4b87e5f6-90fc-4891-aa11-feb499ce5511"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:03:22.146283 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.146261 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b87e5f6-90fc-4891-aa11-feb499ce5511-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4b87e5f6-90fc-4891-aa11-feb499ce5511" (UID: "4b87e5f6-90fc-4891-aa11-feb499ce5511"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:03:22.238141 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.238100 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87r5w\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-kube-api-access-87r5w\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:03:22.238141 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.238136 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4b87e5f6-90fc-4891-aa11-feb499ce5511-trusted-ca\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:03:22.238391 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.238146 2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4b87e5f6-90fc-4891-aa11-feb499ce5511-ca-trust-extracted\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:03:22.238391 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.238185 2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-certificates\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:03:22.238391 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.238198 2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4b87e5f6-90fc-4891-aa11-feb499ce5511-image-registry-private-configuration\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:03:22.238391 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.238207 2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4b87e5f6-90fc-4891-aa11-feb499ce5511-installation-pull-secrets\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:03:22.238391 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.238215 2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-bound-sa-token\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:03:22.238391 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.238223 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4b87e5f6-90fc-4891-aa11-feb499ce5511-registry-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:03:22.601851 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.601816 2564 generic.go:356] "Generic (PLEG): container finished" podID="4b87e5f6-90fc-4891-aa11-feb499ce5511" containerID="4ac9a49d7df276c2d12dd7d2c274a1dcbab89791c6147576d8dfbb18f2045cf1" exitCode=0 Sep 07 13:03:22.602043 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.601863 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-864459d56-bbg5b" Sep 07 13:03:22.602043 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.601888 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864459d56-bbg5b" event={"ID":"4b87e5f6-90fc-4891-aa11-feb499ce5511","Type":"ContainerDied","Data":"4ac9a49d7df276c2d12dd7d2c274a1dcbab89791c6147576d8dfbb18f2045cf1"} Sep 07 13:03:22.602043 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.601926 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-864459d56-bbg5b" event={"ID":"4b87e5f6-90fc-4891-aa11-feb499ce5511","Type":"ContainerDied","Data":"2fb41ab912fbaa3da9ad28b0424b673551714aa727a0e3c5048e34c2722993bd"} Sep 07 13:03:22.602043 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.601946 2564 scope.go:117] "RemoveContainer" containerID="4ac9a49d7df276c2d12dd7d2c274a1dcbab89791c6147576d8dfbb18f2045cf1" Sep 07 13:03:22.616452 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.616431 2564 scope.go:117] "RemoveContainer" containerID="4ac9a49d7df276c2d12dd7d2c274a1dcbab89791c6147576d8dfbb18f2045cf1" Sep 07 13:03:22.616742 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:03:22.616722 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ac9a49d7df276c2d12dd7d2c274a1dcbab89791c6147576d8dfbb18f2045cf1\": container with ID starting with 4ac9a49d7df276c2d12dd7d2c274a1dcbab89791c6147576d8dfbb18f2045cf1 not found: ID does not exist" containerID="4ac9a49d7df276c2d12dd7d2c274a1dcbab89791c6147576d8dfbb18f2045cf1" Sep 07 13:03:22.616792 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.616751 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ac9a49d7df276c2d12dd7d2c274a1dcbab89791c6147576d8dfbb18f2045cf1"} err="failed to get container status \"4ac9a49d7df276c2d12dd7d2c274a1dcbab89791c6147576d8dfbb18f2045cf1\": rpc error: code = NotFound desc = could not find container \"4ac9a49d7df276c2d12dd7d2c274a1dcbab89791c6147576d8dfbb18f2045cf1\": container with ID starting with 4ac9a49d7df276c2d12dd7d2c274a1dcbab89791c6147576d8dfbb18f2045cf1 not found: ID does not exist" Sep 07 13:03:22.628989 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.628967 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-864459d56-bbg5b"] Sep 07 13:03:22.633779 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:22.633760 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-864459d56-bbg5b"] Sep 07 13:03:23.921648 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:23.921615 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b87e5f6-90fc-4891-aa11-feb499ce5511" path="/var/lib/kubelet/pods/4b87e5f6-90fc-4891-aa11-feb499ce5511/volumes" Sep 07 13:03:30.638137 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:30.638098 2564 generic.go:356] "Generic (PLEG): container finished" podID="c7991f8f-dd41-437e-8d24-8d2b3c756304" containerID="c7e44c455378af2eb7f743d8cfedb8c79b39509079847b5fae0aa6573e9c7cf5" exitCode=0 Sep 07 13:03:30.638572 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:30.638169 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w" event={"ID":"c7991f8f-dd41-437e-8d24-8d2b3c756304","Type":"ContainerDied","Data":"c7e44c455378af2eb7f743d8cfedb8c79b39509079847b5fae0aa6573e9c7cf5"} Sep 07 13:03:30.638572 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:30.638477 2564 scope.go:117] "RemoveContainer" containerID="c7e44c455378af2eb7f743d8cfedb8c79b39509079847b5fae0aa6573e9c7cf5" Sep 07 13:03:31.644417 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:31.644377 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-2qm2w" event={"ID":"c7991f8f-dd41-437e-8d24-8d2b3c756304","Type":"ContainerStarted","Data":"38348b4be4b350f49b6169d66618e154bd0e0fd3e06e5ed606c0945e651745b0"} Sep 07 13:03:40.361420 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.361354 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76f6584dc7-spwmw" podUID="d367bdb3-534a-4216-8fc7-16f2dee0d838" containerName="console" containerID="cri-o://49217c7522e2da957c0491229ab6c71ef634727692707620e01df12f208fa16e" gracePeriod=15 Sep 07 13:03:40.629686 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.629626 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:03:40.673501 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.673464 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-serving-cert\") pod \"d367bdb3-534a-4216-8fc7-16f2dee0d838\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " Sep 07 13:03:40.673658 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.673540 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jj7vj\" (UniqueName: \"kubernetes.io/projected/d367bdb3-534a-4216-8fc7-16f2dee0d838-kube-api-access-jj7vj\") pod \"d367bdb3-534a-4216-8fc7-16f2dee0d838\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " Sep 07 13:03:40.673658 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.673563 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-oauth-serving-cert\") pod \"d367bdb3-534a-4216-8fc7-16f2dee0d838\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " Sep 07 13:03:40.673658 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.673622 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-service-ca\") pod \"d367bdb3-534a-4216-8fc7-16f2dee0d838\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " Sep 07 13:03:40.673790 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.673667 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-config\") pod \"d367bdb3-534a-4216-8fc7-16f2dee0d838\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " Sep 07 13:03:40.673790 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.673724 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-oauth-config\") pod \"d367bdb3-534a-4216-8fc7-16f2dee0d838\" (UID: \"d367bdb3-534a-4216-8fc7-16f2dee0d838\") " Sep 07 13:03:40.673967 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.673944 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d367bdb3-534a-4216-8fc7-16f2dee0d838" (UID: "d367bdb3-534a-4216-8fc7-16f2dee0d838"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:03:40.674042 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.673996 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-service-ca" (OuterVolumeSpecName: "service-ca") pod "d367bdb3-534a-4216-8fc7-16f2dee0d838" (UID: "d367bdb3-534a-4216-8fc7-16f2dee0d838"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:03:40.674134 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.674091 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-config" (OuterVolumeSpecName: "console-config") pod "d367bdb3-534a-4216-8fc7-16f2dee0d838" (UID: "d367bdb3-534a-4216-8fc7-16f2dee0d838"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:03:40.675836 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.675812 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d367bdb3-534a-4216-8fc7-16f2dee0d838" (UID: "d367bdb3-534a-4216-8fc7-16f2dee0d838"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:03:40.675931 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.675877 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d367bdb3-534a-4216-8fc7-16f2dee0d838-kube-api-access-jj7vj" (OuterVolumeSpecName: "kube-api-access-jj7vj") pod "d367bdb3-534a-4216-8fc7-16f2dee0d838" (UID: "d367bdb3-534a-4216-8fc7-16f2dee0d838"). InnerVolumeSpecName "kube-api-access-jj7vj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:03:40.675984 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.675923 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d367bdb3-534a-4216-8fc7-16f2dee0d838" (UID: "d367bdb3-534a-4216-8fc7-16f2dee0d838"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:03:40.680679 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.680653 2564 generic.go:356] "Generic (PLEG): container finished" podID="d367bdb3-534a-4216-8fc7-16f2dee0d838" containerID="49217c7522e2da957c0491229ab6c71ef634727692707620e01df12f208fa16e" exitCode=0 Sep 07 13:03:40.680765 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.680719 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76f6584dc7-spwmw" Sep 07 13:03:40.680765 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.680740 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76f6584dc7-spwmw" event={"ID":"d367bdb3-534a-4216-8fc7-16f2dee0d838","Type":"ContainerDied","Data":"49217c7522e2da957c0491229ab6c71ef634727692707620e01df12f208fa16e"} Sep 07 13:03:40.680880 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.680837 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76f6584dc7-spwmw" event={"ID":"d367bdb3-534a-4216-8fc7-16f2dee0d838","Type":"ContainerDied","Data":"3d6e2d867341a38742ac827863a4345ff34fa2861dfe63b7883a5a12399980da"} Sep 07 13:03:40.680880 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.680863 2564 scope.go:117] "RemoveContainer" containerID="49217c7522e2da957c0491229ab6c71ef634727692707620e01df12f208fa16e" Sep 07 13:03:40.691271 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.691251 2564 scope.go:117] "RemoveContainer" containerID="49217c7522e2da957c0491229ab6c71ef634727692707620e01df12f208fa16e" Sep 07 13:03:40.691550 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:03:40.691531 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49217c7522e2da957c0491229ab6c71ef634727692707620e01df12f208fa16e\": container with ID starting with 49217c7522e2da957c0491229ab6c71ef634727692707620e01df12f208fa16e not found: ID does not exist" containerID="49217c7522e2da957c0491229ab6c71ef634727692707620e01df12f208fa16e" Sep 07 13:03:40.691594 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.691558 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49217c7522e2da957c0491229ab6c71ef634727692707620e01df12f208fa16e"} err="failed to get container status \"49217c7522e2da957c0491229ab6c71ef634727692707620e01df12f208fa16e\": rpc error: code = NotFound desc = could not find container \"49217c7522e2da957c0491229ab6c71ef634727692707620e01df12f208fa16e\": container with ID starting with 49217c7522e2da957c0491229ab6c71ef634727692707620e01df12f208fa16e not found: ID does not exist" Sep 07 13:03:40.703259 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.703235 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76f6584dc7-spwmw"] Sep 07 13:03:40.706785 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.706763 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76f6584dc7-spwmw"] Sep 07 13:03:40.775119 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.775085 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jj7vj\" (UniqueName: \"kubernetes.io/projected/d367bdb3-534a-4216-8fc7-16f2dee0d838-kube-api-access-jj7vj\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:03:40.775119 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.775114 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-oauth-serving-cert\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:03:40.775119 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.775127 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-service-ca\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:03:40.775345 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.775136 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-config\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:03:40.775345 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.775144 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-oauth-config\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:03:40.775345 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:40.775152 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d367bdb3-534a-4216-8fc7-16f2dee0d838-console-serving-cert\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:03:41.921413 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:03:41.921380 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d367bdb3-534a-4216-8fc7-16f2dee0d838" path="/var/lib/kubelet/pods/d367bdb3-534a-4216-8fc7-16f2dee0d838/volumes" Sep 07 13:04:06.293277 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:06.293226 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 13:04:06.293784 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:06.293548 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="alertmanager" containerID="cri-o://40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c" gracePeriod=120 Sep 07 13:04:06.293784 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:06.293588 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="kube-rbac-proxy-metric" containerID="cri-o://bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0" gracePeriod=120 Sep 07 13:04:06.293784 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:06.293623 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="kube-rbac-proxy-web" containerID="cri-o://caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a" gracePeriod=120 Sep 07 13:04:06.293784 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:06.293629 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="config-reloader" containerID="cri-o://8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357" gracePeriod=120 Sep 07 13:04:06.293784 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:06.293659 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="kube-rbac-proxy" containerID="cri-o://b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626" gracePeriod=120 Sep 07 13:04:06.293784 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:06.293682 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="prom-label-proxy" containerID="cri-o://f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc" gracePeriod=120 Sep 07 13:04:06.784524 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:06.784490 2564 generic.go:356] "Generic (PLEG): container finished" podID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerID="f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc" exitCode=0 Sep 07 13:04:06.784524 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:06.784519 2564 generic.go:356] "Generic (PLEG): container finished" podID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerID="b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626" exitCode=0 Sep 07 13:04:06.784524 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:06.784525 2564 generic.go:356] "Generic (PLEG): container finished" podID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerID="8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357" exitCode=0 Sep 07 13:04:06.784524 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:06.784532 2564 generic.go:356] "Generic (PLEG): container finished" podID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerID="40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c" exitCode=0 Sep 07 13:04:06.784798 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:06.784567 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d0cb95c-fbda-4e94-8d16-46a6c197b822","Type":"ContainerDied","Data":"f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc"} Sep 07 13:04:06.784798 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:06.784632 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d0cb95c-fbda-4e94-8d16-46a6c197b822","Type":"ContainerDied","Data":"b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626"} Sep 07 13:04:06.784798 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:06.784661 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d0cb95c-fbda-4e94-8d16-46a6c197b822","Type":"ContainerDied","Data":"8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357"} Sep 07 13:04:06.784798 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:06.784684 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d0cb95c-fbda-4e94-8d16-46a6c197b822","Type":"ContainerDied","Data":"40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c"} Sep 07 13:04:07.554387 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.554355 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:07.694641 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.694539 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d0cb95c-fbda-4e94-8d16-46a6c197b822-metrics-client-ca\") pod \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " Sep 07 13:04:07.694641 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.694595 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6d0cb95c-fbda-4e94-8d16-46a6c197b822-alertmanager-main-db\") pod \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " Sep 07 13:04:07.694641 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.694631 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d0cb95c-fbda-4e94-8d16-46a6c197b822-alertmanager-trusted-ca-bundle\") pod \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " Sep 07 13:04:07.694924 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.694657 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6d0cb95c-fbda-4e94-8d16-46a6c197b822-config-out\") pod \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " Sep 07 13:04:07.694924 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.694680 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy-web\") pod \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " Sep 07 13:04:07.694924 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.694704 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-config-volume\") pod \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " Sep 07 13:04:07.694924 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.694743 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xd8sd\" (UniqueName: \"kubernetes.io/projected/6d0cb95c-fbda-4e94-8d16-46a6c197b822-kube-api-access-xd8sd\") pod \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " Sep 07 13:04:07.694924 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.694793 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy\") pod \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " Sep 07 13:04:07.694924 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.694817 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-web-config\") pod \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " Sep 07 13:04:07.694924 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.694853 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-cluster-tls-config\") pod \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " Sep 07 13:04:07.694924 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.694882 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy-metric\") pod \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " Sep 07 13:04:07.694924 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.694919 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-main-tls\") pod \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " Sep 07 13:04:07.695377 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.694955 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6d0cb95c-fbda-4e94-8d16-46a6c197b822-tls-assets\") pod \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\" (UID: \"6d0cb95c-fbda-4e94-8d16-46a6c197b822\") " Sep 07 13:04:07.695377 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.695002 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d0cb95c-fbda-4e94-8d16-46a6c197b822-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "6d0cb95c-fbda-4e94-8d16-46a6c197b822" (UID: "6d0cb95c-fbda-4e94-8d16-46a6c197b822"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:04:07.695377 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.695078 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d0cb95c-fbda-4e94-8d16-46a6c197b822-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "6d0cb95c-fbda-4e94-8d16-46a6c197b822" (UID: "6d0cb95c-fbda-4e94-8d16-46a6c197b822"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:04:07.695377 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.695207 2564 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d0cb95c-fbda-4e94-8d16-46a6c197b822-metrics-client-ca\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:04:07.695377 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.695225 2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6d0cb95c-fbda-4e94-8d16-46a6c197b822-alertmanager-main-db\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:04:07.695644 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.695430 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d0cb95c-fbda-4e94-8d16-46a6c197b822-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "6d0cb95c-fbda-4e94-8d16-46a6c197b822" (UID: "6d0cb95c-fbda-4e94-8d16-46a6c197b822"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:04:07.697688 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.697654 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "6d0cb95c-fbda-4e94-8d16-46a6c197b822" (UID: "6d0cb95c-fbda-4e94-8d16-46a6c197b822"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:04:07.698170 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.698127 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-config-volume" (OuterVolumeSpecName: "config-volume") pod "6d0cb95c-fbda-4e94-8d16-46a6c197b822" (UID: "6d0cb95c-fbda-4e94-8d16-46a6c197b822"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:04:07.698479 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.698441 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "6d0cb95c-fbda-4e94-8d16-46a6c197b822" (UID: "6d0cb95c-fbda-4e94-8d16-46a6c197b822"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:04:07.698682 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.698644 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "6d0cb95c-fbda-4e94-8d16-46a6c197b822" (UID: "6d0cb95c-fbda-4e94-8d16-46a6c197b822"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:04:07.698682 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.698648 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d0cb95c-fbda-4e94-8d16-46a6c197b822-kube-api-access-xd8sd" (OuterVolumeSpecName: "kube-api-access-xd8sd") pod "6d0cb95c-fbda-4e94-8d16-46a6c197b822" (UID: "6d0cb95c-fbda-4e94-8d16-46a6c197b822"). InnerVolumeSpecName "kube-api-access-xd8sd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:04:07.698840 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.698812 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d0cb95c-fbda-4e94-8d16-46a6c197b822-config-out" (OuterVolumeSpecName: "config-out") pod "6d0cb95c-fbda-4e94-8d16-46a6c197b822" (UID: "6d0cb95c-fbda-4e94-8d16-46a6c197b822"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:04:07.699416 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.699387 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "6d0cb95c-fbda-4e94-8d16-46a6c197b822" (UID: "6d0cb95c-fbda-4e94-8d16-46a6c197b822"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:04:07.699499 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.699427 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d0cb95c-fbda-4e94-8d16-46a6c197b822-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "6d0cb95c-fbda-4e94-8d16-46a6c197b822" (UID: "6d0cb95c-fbda-4e94-8d16-46a6c197b822"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:04:07.702664 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.702639 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "6d0cb95c-fbda-4e94-8d16-46a6c197b822" (UID: "6d0cb95c-fbda-4e94-8d16-46a6c197b822"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:04:07.708956 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.708934 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-web-config" (OuterVolumeSpecName: "web-config") pod "6d0cb95c-fbda-4e94-8d16-46a6c197b822" (UID: "6d0cb95c-fbda-4e94-8d16-46a6c197b822"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:04:07.791431 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.791399 2564 generic.go:356] "Generic (PLEG): container finished" podID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerID="bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0" exitCode=0 Sep 07 13:04:07.791431 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.791427 2564 generic.go:356] "Generic (PLEG): container finished" podID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerID="caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a" exitCode=0 Sep 07 13:04:07.791649 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.791464 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d0cb95c-fbda-4e94-8d16-46a6c197b822","Type":"ContainerDied","Data":"bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0"} Sep 07 13:04:07.791649 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.791473 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:07.791649 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.791543 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d0cb95c-fbda-4e94-8d16-46a6c197b822","Type":"ContainerDied","Data":"caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a"} Sep 07 13:04:07.791649 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.791584 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d0cb95c-fbda-4e94-8d16-46a6c197b822","Type":"ContainerDied","Data":"cb4ee053a7d794577aa2493cdf39fde52642bf74920e6e3d9414025d485ba1db"} Sep 07 13:04:07.791649 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.791616 2564 scope.go:117] "RemoveContainer" containerID="f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc" Sep 07 13:04:07.795595 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.795574 2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d0cb95c-fbda-4e94-8d16-46a6c197b822-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:04:07.795684 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.795598 2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6d0cb95c-fbda-4e94-8d16-46a6c197b822-config-out\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:04:07.795684 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.795613 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:04:07.795684 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.795626 2564 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-config-volume\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:04:07.795684 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.795639 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xd8sd\" (UniqueName: \"kubernetes.io/projected/6d0cb95c-fbda-4e94-8d16-46a6c197b822-kube-api-access-xd8sd\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:04:07.795684 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.795652 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:04:07.795684 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.795667 2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-web-config\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:04:07.795684 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.795678 2564 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-cluster-tls-config\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:04:07.795980 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.795691 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:04:07.795980 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.795706 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6d0cb95c-fbda-4e94-8d16-46a6c197b822-secret-alertmanager-main-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:04:07.795980 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.795718 2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6d0cb95c-fbda-4e94-8d16-46a6c197b822-tls-assets\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:04:07.799424 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.799342 2564 scope.go:117] "RemoveContainer" containerID="bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0" Sep 07 13:04:07.806177 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.806166 2564 scope.go:117] "RemoveContainer" containerID="b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626" Sep 07 13:04:07.812948 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.812930 2564 scope.go:117] "RemoveContainer" containerID="caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a" Sep 07 13:04:07.820456 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.820380 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 13:04:07.820509 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.820498 2564 scope.go:117] "RemoveContainer" containerID="8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357" Sep 07 13:04:07.824969 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.824945 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 13:04:07.827823 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.827808 2564 scope.go:117] "RemoveContainer" containerID="40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c" Sep 07 13:04:07.834786 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.834769 2564 scope.go:117] "RemoveContainer" containerID="5cce426dbc9feb788549b269b1f38e2cce9f35d5fc62cccf378ec540a1eca267" Sep 07 13:04:07.841407 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.841384 2564 scope.go:117] "RemoveContainer" containerID="f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc" Sep 07 13:04:07.841663 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:04:07.841642 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc\": container with ID starting with f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc not found: ID does not exist" containerID="f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc" Sep 07 13:04:07.841734 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.841676 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc"} err="failed to get container status \"f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc\": rpc error: code = NotFound desc = could not find container \"f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc\": container with ID starting with f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc not found: ID does not exist" Sep 07 13:04:07.841734 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.841702 2564 scope.go:117] "RemoveContainer" containerID="bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0" Sep 07 13:04:07.841959 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:04:07.841942 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0\": container with ID starting with bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0 not found: ID does not exist" containerID="bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0" Sep 07 13:04:07.842018 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.841964 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0"} err="failed to get container status \"bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0\": rpc error: code = NotFound desc = could not find container \"bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0\": container with ID starting with bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0 not found: ID does not exist" Sep 07 13:04:07.842018 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.841980 2564 scope.go:117] "RemoveContainer" containerID="b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626" Sep 07 13:04:07.842215 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:04:07.842200 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626\": container with ID starting with b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626 not found: ID does not exist" containerID="b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626" Sep 07 13:04:07.842255 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.842220 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626"} err="failed to get container status \"b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626\": rpc error: code = NotFound desc = could not find container \"b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626\": container with ID starting with b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626 not found: ID does not exist" Sep 07 13:04:07.842255 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.842235 2564 scope.go:117] "RemoveContainer" containerID="caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a" Sep 07 13:04:07.842486 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:04:07.842469 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a\": container with ID starting with caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a not found: ID does not exist" containerID="caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a" Sep 07 13:04:07.842537 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.842494 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a"} err="failed to get container status \"caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a\": rpc error: code = NotFound desc = could not find container \"caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a\": container with ID starting with caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a not found: ID does not exist" Sep 07 13:04:07.842537 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.842511 2564 scope.go:117] "RemoveContainer" containerID="8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357" Sep 07 13:04:07.842762 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:04:07.842746 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357\": container with ID starting with 8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357 not found: ID does not exist" containerID="8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357" Sep 07 13:04:07.842811 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.842764 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357"} err="failed to get container status \"8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357\": rpc error: code = NotFound desc = could not find container \"8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357\": container with ID starting with 8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357 not found: ID does not exist" Sep 07 13:04:07.842811 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.842776 2564 scope.go:117] "RemoveContainer" containerID="40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c" Sep 07 13:04:07.843013 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:04:07.842997 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c\": container with ID starting with 40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c not found: ID does not exist" containerID="40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c" Sep 07 13:04:07.843071 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.843018 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c"} err="failed to get container status \"40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c\": rpc error: code = NotFound desc = could not find container \"40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c\": container with ID starting with 40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c not found: ID does not exist" Sep 07 13:04:07.843071 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.843031 2564 scope.go:117] "RemoveContainer" containerID="5cce426dbc9feb788549b269b1f38e2cce9f35d5fc62cccf378ec540a1eca267" Sep 07 13:04:07.843361 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:04:07.843342 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cce426dbc9feb788549b269b1f38e2cce9f35d5fc62cccf378ec540a1eca267\": container with ID starting with 5cce426dbc9feb788549b269b1f38e2cce9f35d5fc62cccf378ec540a1eca267 not found: ID does not exist" containerID="5cce426dbc9feb788549b269b1f38e2cce9f35d5fc62cccf378ec540a1eca267" Sep 07 13:04:07.843419 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.843363 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cce426dbc9feb788549b269b1f38e2cce9f35d5fc62cccf378ec540a1eca267"} err="failed to get container status \"5cce426dbc9feb788549b269b1f38e2cce9f35d5fc62cccf378ec540a1eca267\": rpc error: code = NotFound desc = could not find container \"5cce426dbc9feb788549b269b1f38e2cce9f35d5fc62cccf378ec540a1eca267\": container with ID starting with 5cce426dbc9feb788549b269b1f38e2cce9f35d5fc62cccf378ec540a1eca267 not found: ID does not exist" Sep 07 13:04:07.843419 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.843382 2564 scope.go:117] "RemoveContainer" containerID="f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc" Sep 07 13:04:07.843627 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.843609 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc"} err="failed to get container status \"f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc\": rpc error: code = NotFound desc = could not find container \"f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc\": container with ID starting with f94ca35319484aae3b07ec79b66a240677b6df37dd264f21af1b30d8e5c4ffcc not found: ID does not exist" Sep 07 13:04:07.843664 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.843628 2564 scope.go:117] "RemoveContainer" containerID="bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0" Sep 07 13:04:07.843846 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.843817 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0"} err="failed to get container status \"bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0\": rpc error: code = NotFound desc = could not find container \"bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0\": container with ID starting with bc0dbf288df2fe8ed94a091c9b52d2bf341ed972e4cc24169ed7488bbda9fdd0 not found: ID does not exist" Sep 07 13:04:07.843897 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.843846 2564 scope.go:117] "RemoveContainer" containerID="b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626" Sep 07 13:04:07.844094 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.844077 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626"} err="failed to get container status \"b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626\": rpc error: code = NotFound desc = could not find container \"b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626\": container with ID starting with b728a1d162b4b36940ec94c220e23481b60242795395508d9a51d00af4051626 not found: ID does not exist" Sep 07 13:04:07.844132 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.844096 2564 scope.go:117] "RemoveContainer" containerID="caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a" Sep 07 13:04:07.844337 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.844320 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a"} err="failed to get container status \"caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a\": rpc error: code = NotFound desc = could not find container \"caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a\": container with ID starting with caadc706d72c57b3ff294acde6a279ea6fb70981ea861849ed353d73bdca5e6a not found: ID does not exist" Sep 07 13:04:07.844375 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.844338 2564 scope.go:117] "RemoveContainer" containerID="8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357" Sep 07 13:04:07.844597 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.844581 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357"} err="failed to get container status \"8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357\": rpc error: code = NotFound desc = could not find container \"8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357\": container with ID starting with 8635150a90a1b10e0bbcd17f05d43923bdfa2b5f9a78e35bf3e068ad4ec63357 not found: ID does not exist" Sep 07 13:04:07.844637 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.844599 2564 scope.go:117] "RemoveContainer" containerID="40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c" Sep 07 13:04:07.844790 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.844770 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c"} err="failed to get container status \"40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c\": rpc error: code = NotFound desc = could not find container \"40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c\": container with ID starting with 40bdd802b8aa30f1f6942b0ae82a686a36eeedb522a2f3202071d7f98074f46c not found: ID does not exist" Sep 07 13:04:07.844828 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.844792 2564 scope.go:117] "RemoveContainer" containerID="5cce426dbc9feb788549b269b1f38e2cce9f35d5fc62cccf378ec540a1eca267" Sep 07 13:04:07.844987 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.844970 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cce426dbc9feb788549b269b1f38e2cce9f35d5fc62cccf378ec540a1eca267"} err="failed to get container status \"5cce426dbc9feb788549b269b1f38e2cce9f35d5fc62cccf378ec540a1eca267\": rpc error: code = NotFound desc = could not find container \"5cce426dbc9feb788549b269b1f38e2cce9f35d5fc62cccf378ec540a1eca267\": container with ID starting with 5cce426dbc9feb788549b269b1f38e2cce9f35d5fc62cccf378ec540a1eca267 not found: ID does not exist" Sep 07 13:04:07.861421 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861397 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 13:04:07.861675 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861662 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="kube-rbac-proxy" Sep 07 13:04:07.861718 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861680 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="kube-rbac-proxy" Sep 07 13:04:07.861718 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861692 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="kube-rbac-proxy-web" Sep 07 13:04:07.861718 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861698 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="kube-rbac-proxy-web" Sep 07 13:04:07.861718 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861704 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="config-reloader" Sep 07 13:04:07.861718 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861709 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="config-reloader" Sep 07 13:04:07.861718 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861718 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="init-config-reloader" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861724 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="init-config-reloader" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861732 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="kube-rbac-proxy-metric" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861737 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="kube-rbac-proxy-metric" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861750 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b87e5f6-90fc-4891-aa11-feb499ce5511" containerName="registry" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861758 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b87e5f6-90fc-4891-aa11-feb499ce5511" containerName="registry" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861772 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="alertmanager" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861777 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="alertmanager" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861784 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d367bdb3-534a-4216-8fc7-16f2dee0d838" containerName="console" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861789 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d367bdb3-534a-4216-8fc7-16f2dee0d838" containerName="console" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861795 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="prom-label-proxy" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861800 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="prom-label-proxy" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861848 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="kube-rbac-proxy" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861856 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="kube-rbac-proxy-metric" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861861 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d367bdb3-534a-4216-8fc7-16f2dee0d838" containerName="console" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861867 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b87e5f6-90fc-4891-aa11-feb499ce5511" containerName="registry" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861873 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="alertmanager" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861877 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="prom-label-proxy" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861883 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="kube-rbac-proxy-web" Sep 07 13:04:07.861883 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.861890 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" containerName="config-reloader" Sep 07 13:04:07.866869 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.866851 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:07.872130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.871373 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 07 13:04:07.872130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.871919 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 07 13:04:07.873331 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.872177 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 07 13:04:07.873331 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.873171 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 07 13:04:07.873331 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.873194 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 07 13:04:07.873509 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.873398 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 07 13:04:07.873509 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.873490 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9kr8m\"" Sep 07 13:04:07.873682 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.873662 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 07 13:04:07.873758 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.873702 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 07 13:04:07.878693 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.878677 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 07 13:04:07.884123 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.884103 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 13:04:07.925808 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.925771 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d0cb95c-fbda-4e94-8d16-46a6c197b822" path="/var/lib/kubelet/pods/6d0cb95c-fbda-4e94-8d16-46a6c197b822/volumes" Sep 07 13:04:07.996881 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.996849 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f88d5418-255b-48e8-822f-8cb0ff7ee119-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:07.997042 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.996889 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:07.997042 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.996915 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:07.997042 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.997005 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:07.997223 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.997075 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f88d5418-255b-48e8-822f-8cb0ff7ee119-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:07.997223 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.997100 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx7hp\" (UniqueName: \"kubernetes.io/projected/f88d5418-255b-48e8-822f-8cb0ff7ee119-kube-api-access-fx7hp\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:07.997223 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.997132 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-web-config\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:07.997223 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.997189 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f88d5418-255b-48e8-822f-8cb0ff7ee119-config-out\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:07.997418 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.997232 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:07.997418 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.997269 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88d5418-255b-48e8-822f-8cb0ff7ee119-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:07.997418 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.997297 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f88d5418-255b-48e8-822f-8cb0ff7ee119-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:07.997418 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.997325 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-config-volume\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:07.997418 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:07.997349 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.098535 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.098503 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f88d5418-255b-48e8-822f-8cb0ff7ee119-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.098535 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.098541 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.098749 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.098561 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.098749 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.098585 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.098749 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.098609 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f88d5418-255b-48e8-822f-8cb0ff7ee119-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.098749 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.098624 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx7hp\" (UniqueName: \"kubernetes.io/projected/f88d5418-255b-48e8-822f-8cb0ff7ee119-kube-api-access-fx7hp\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.098749 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.098642 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-web-config\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.098749 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.098657 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f88d5418-255b-48e8-822f-8cb0ff7ee119-config-out\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.098749 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.098680 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.098749 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.098712 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88d5418-255b-48e8-822f-8cb0ff7ee119-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.099137 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.099029 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f88d5418-255b-48e8-822f-8cb0ff7ee119-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.099137 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.099093 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-config-volume\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.099137 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.099123 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.099297 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.099142 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f88d5418-255b-48e8-822f-8cb0ff7ee119-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.099692 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.099664 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88d5418-255b-48e8-822f-8cb0ff7ee119-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.099866 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.099776 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f88d5418-255b-48e8-822f-8cb0ff7ee119-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.101607 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.101572 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f88d5418-255b-48e8-822f-8cb0ff7ee119-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.101607 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.101595 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.101752 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.101728 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-web-config\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.101820 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.101779 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.101932 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.101913 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.102207 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.102185 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-config-volume\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.102303 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.102240 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f88d5418-255b-48e8-822f-8cb0ff7ee119-config-out\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.102565 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.102548 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.103357 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.103337 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f88d5418-255b-48e8-822f-8cb0ff7ee119-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.106994 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.106978 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx7hp\" (UniqueName: \"kubernetes.io/projected/f88d5418-255b-48e8-822f-8cb0ff7ee119-kube-api-access-fx7hp\") pod \"alertmanager-main-0\" (UID: \"f88d5418-255b-48e8-822f-8cb0ff7ee119\") " pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.177876 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.177824 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 07 13:04:08.461832 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.461806 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 07 13:04:08.463779 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:04:08.463749 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf88d5418_255b_48e8_822f_8cb0ff7ee119.slice/crio-ad7c9391a1fede765325282a5f8ce712938d95688cd828518af798474808c547 WatchSource:0}: Error finding container ad7c9391a1fede765325282a5f8ce712938d95688cd828518af798474808c547: Status 404 returned error can't find the container with id ad7c9391a1fede765325282a5f8ce712938d95688cd828518af798474808c547 Sep 07 13:04:08.797871 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.797836 2564 generic.go:356] "Generic (PLEG): container finished" podID="f88d5418-255b-48e8-822f-8cb0ff7ee119" containerID="a4791aaa9ad202202c6ffb03b9e3c413f81f8b2352689a4f90259d952964399b" exitCode=0 Sep 07 13:04:08.798276 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.797921 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f88d5418-255b-48e8-822f-8cb0ff7ee119","Type":"ContainerDied","Data":"a4791aaa9ad202202c6ffb03b9e3c413f81f8b2352689a4f90259d952964399b"} Sep 07 13:04:08.798276 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:08.797986 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f88d5418-255b-48e8-822f-8cb0ff7ee119","Type":"ContainerStarted","Data":"ad7c9391a1fede765325282a5f8ce712938d95688cd828518af798474808c547"} Sep 07 13:04:09.804822 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:09.804785 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f88d5418-255b-48e8-822f-8cb0ff7ee119","Type":"ContainerStarted","Data":"688ec7d271efb88fff646e2ac70df761ab1d27c95754f0986f59a019b994fe21"} Sep 07 13:04:09.805198 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:09.804868 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f88d5418-255b-48e8-822f-8cb0ff7ee119","Type":"ContainerStarted","Data":"d11e799d29caf6ffa4a0deb2e4a2c3c380cd36ead523ea20279f2bc9970d971e"} Sep 07 13:04:09.805198 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:09.804913 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f88d5418-255b-48e8-822f-8cb0ff7ee119","Type":"ContainerStarted","Data":"cf0f4331257a3eb70c474efa0e6ca32e5fdca5cdef611e4aeba7ec732434c9fd"} Sep 07 13:04:09.805198 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:09.804949 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f88d5418-255b-48e8-822f-8cb0ff7ee119","Type":"ContainerStarted","Data":"640e89015ec083d6d86a7a45c4dd4f3a1bf79c3aa113bb2b275f6781c066e708"} Sep 07 13:04:09.805198 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:09.804988 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f88d5418-255b-48e8-822f-8cb0ff7ee119","Type":"ContainerStarted","Data":"9953a35cad7ec6b74addb1df13f4f630e01d327f0b56cd7ba549e4dfc5fdfe39"} Sep 07 13:04:09.805198 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:09.805023 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f88d5418-255b-48e8-822f-8cb0ff7ee119","Type":"ContainerStarted","Data":"0e3eaa5d4fba7345389afb653246ecca9b75f8dba60ac4bf8998bd299db8a6f8"} Sep 07 13:04:09.839006 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:09.838951 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.8229650680000002 podStartE2EDuration="2.838929599s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.964531ms" isStatefulPod=false podCreationTimestamp="2026-09-07 13:04:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:04:09.836386269 +0000 UTC m=+210.431890885" watchObservedRunningTime="2026-09-07 13:04:09.838929599 +0000 UTC m=+210.434434214" Sep 07 13:04:10.408708 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.408655 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj"] Sep 07 13:04:10.413605 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.413579 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.421461 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.421440 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 07 13:04:10.421578 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.421489 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 07 13:04:10.421806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.421783 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wgwgw\"" Sep 07 13:04:10.421923 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.421825 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 07 13:04:10.421923 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.421829 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 07 13:04:10.422039 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.421928 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 07 13:04:10.446568 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.446544 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 07 13:04:10.450967 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.450942 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj"] Sep 07 13:04:10.520243 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.520201 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce3bfbce-e613-4016-ae00-a3b49a64e500-serving-certs-ca-bundle\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.520397 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.520255 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ce3bfbce-e613-4016-ae00-a3b49a64e500-secret-telemeter-client\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.520397 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.520287 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce3bfbce-e613-4016-ae00-a3b49a64e500-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.520397 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.520327 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ce3bfbce-e613-4016-ae00-a3b49a64e500-federate-client-tls\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.520397 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.520369 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ce3bfbce-e613-4016-ae00-a3b49a64e500-telemeter-client-tls\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.520561 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.520400 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce3bfbce-e613-4016-ae00-a3b49a64e500-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.520561 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.520434 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce3bfbce-e613-4016-ae00-a3b49a64e500-metrics-client-ca\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.520561 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.520475 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slwzn\" (UniqueName: \"kubernetes.io/projected/ce3bfbce-e613-4016-ae00-a3b49a64e500-kube-api-access-slwzn\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.621142 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.621104 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ce3bfbce-e613-4016-ae00-a3b49a64e500-federate-client-tls\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.621142 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.621144 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ce3bfbce-e613-4016-ae00-a3b49a64e500-telemeter-client-tls\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.621364 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.621165 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce3bfbce-e613-4016-ae00-a3b49a64e500-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.621403 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.621364 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce3bfbce-e613-4016-ae00-a3b49a64e500-metrics-client-ca\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.621472 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.621440 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slwzn\" (UniqueName: \"kubernetes.io/projected/ce3bfbce-e613-4016-ae00-a3b49a64e500-kube-api-access-slwzn\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.621589 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.621498 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce3bfbce-e613-4016-ae00-a3b49a64e500-serving-certs-ca-bundle\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.621589 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.621527 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ce3bfbce-e613-4016-ae00-a3b49a64e500-secret-telemeter-client\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.621589 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.621568 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce3bfbce-e613-4016-ae00-a3b49a64e500-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.622280 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.622247 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce3bfbce-e613-4016-ae00-a3b49a64e500-serving-certs-ca-bundle\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.622417 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.622368 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce3bfbce-e613-4016-ae00-a3b49a64e500-metrics-client-ca\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.622417 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.622398 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce3bfbce-e613-4016-ae00-a3b49a64e500-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.623862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.623831 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ce3bfbce-e613-4016-ae00-a3b49a64e500-telemeter-client-tls\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.624175 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.624155 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ce3bfbce-e613-4016-ae00-a3b49a64e500-federate-client-tls\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.624363 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.624343 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce3bfbce-e613-4016-ae00-a3b49a64e500-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.624493 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.624472 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ce3bfbce-e613-4016-ae00-a3b49a64e500-secret-telemeter-client\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.630632 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.630608 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slwzn\" (UniqueName: \"kubernetes.io/projected/ce3bfbce-e613-4016-ae00-a3b49a64e500-kube-api-access-slwzn\") pod \"telemeter-client-85fd5c4f45-mc5hj\" (UID: \"ce3bfbce-e613-4016-ae00-a3b49a64e500\") " pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:10.722162 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:10.722035 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" Sep 07 13:04:11.013727 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:11.013697 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj"] Sep 07 13:04:11.015148 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:04:11.015117 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce3bfbce_e613_4016_ae00_a3b49a64e500.slice/crio-80a2dfe063ac8c8a225553972e80d96f04c1263c95e787dbd32b34b2a1ca119f WatchSource:0}: Error finding container 80a2dfe063ac8c8a225553972e80d96f04c1263c95e787dbd32b34b2a1ca119f: Status 404 returned error can't find the container with id 80a2dfe063ac8c8a225553972e80d96f04c1263c95e787dbd32b34b2a1ca119f Sep 07 13:04:11.816969 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:11.816917 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" event={"ID":"ce3bfbce-e613-4016-ae00-a3b49a64e500","Type":"ContainerStarted","Data":"80a2dfe063ac8c8a225553972e80d96f04c1263c95e787dbd32b34b2a1ca119f"} Sep 07 13:04:13.827489 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:13.827442 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" event={"ID":"ce3bfbce-e613-4016-ae00-a3b49a64e500","Type":"ContainerStarted","Data":"6eb845c957093aaf18edc3c5f2884762dcf7050ee7c2b03427953de95efcf463"} Sep 07 13:04:13.827911 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:13.827520 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" event={"ID":"ce3bfbce-e613-4016-ae00-a3b49a64e500","Type":"ContainerStarted","Data":"fe946ab7e29121f810cf9afdc19bfffbc035dcddb6e686d4a689860b41e81086"} Sep 07 13:04:13.827911 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:13.827549 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" event={"ID":"ce3bfbce-e613-4016-ae00-a3b49a64e500","Type":"ContainerStarted","Data":"f6ce56180fa9f57b71e976ddd0c448842bd49be6c1e1860a54274072a7a79494"} Sep 07 13:04:13.859815 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:13.859762 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-85fd5c4f45-mc5hj" podStartSLOduration=1.9662380000000002 podStartE2EDuration="3.859745866s" totalImagesPullingTime="1.893507866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:04:13.858334927 +0000 UTC m=+214.453839546" watchObservedRunningTime="2026-09-07 13:04:13.859745866 +0000 UTC m=+214.455250524" Sep 07 13:04:13.963345 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:13.963312 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b8ddbf7db-ck7cf"] Sep 07 13:04:13.969813 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:13.969783 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:13.973772 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:13.973748 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 07 13:04:13.973895 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:13.973824 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 07 13:04:13.974253 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:13.974234 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 07 13:04:13.974625 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:13.974606 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 07 13:04:13.974871 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:13.974849 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jwmrq\"" Sep 07 13:04:13.979822 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:13.979768 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 07 13:04:13.981451 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:13.981429 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b8ddbf7db-ck7cf"] Sep 07 13:04:13.985552 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:13.985532 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 07 13:04:14.051859 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.051821 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c987d10-8899-411d-966d-702b918806c7-console-serving-cert\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.052073 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.051883 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-trusted-ca-bundle\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.052073 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.051946 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w4x8\" (UniqueName: \"kubernetes.io/projected/4c987d10-8899-411d-966d-702b918806c7-kube-api-access-6w4x8\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.052073 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.052011 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-oauth-serving-cert\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.052073 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.052064 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-service-ca\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.052249 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.052100 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c987d10-8899-411d-966d-702b918806c7-console-oauth-config\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.052249 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.052125 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-console-config\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.152788 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.152689 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c987d10-8899-411d-966d-702b918806c7-console-oauth-config\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.152788 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.152731 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-console-config\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.152788 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.152766 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c987d10-8899-411d-966d-702b918806c7-console-serving-cert\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.152788 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.152794 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-trusted-ca-bundle\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.153158 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.152815 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w4x8\" (UniqueName: \"kubernetes.io/projected/4c987d10-8899-411d-966d-702b918806c7-kube-api-access-6w4x8\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.153158 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.152838 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-oauth-serving-cert\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.153158 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.152865 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-service-ca\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.153647 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.153616 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-service-ca\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.153770 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.153619 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-console-config\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.153770 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.153748 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-oauth-serving-cert\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.153887 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.153748 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-trusted-ca-bundle\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.155311 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.155291 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c987d10-8899-411d-966d-702b918806c7-console-oauth-config\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.155525 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.155501 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c987d10-8899-411d-966d-702b918806c7-console-serving-cert\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.162618 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.162593 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w4x8\" (UniqueName: \"kubernetes.io/projected/4c987d10-8899-411d-966d-702b918806c7-kube-api-access-6w4x8\") pod \"console-5b8ddbf7db-ck7cf\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.279684 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.279648 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:14.561766 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.561735 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b8ddbf7db-ck7cf"] Sep 07 13:04:14.565206 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:04:14.565174 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c987d10_8899_411d_966d_702b918806c7.slice/crio-ec7ba81d04f3f4f410d884ef0b0c12cc259ba8205697c76015e1877a42974d0a WatchSource:0}: Error finding container ec7ba81d04f3f4f410d884ef0b0c12cc259ba8205697c76015e1877a42974d0a: Status 404 returned error can't find the container with id ec7ba81d04f3f4f410d884ef0b0c12cc259ba8205697c76015e1877a42974d0a Sep 07 13:04:14.835929 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.835841 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b8ddbf7db-ck7cf" event={"ID":"4c987d10-8899-411d-966d-702b918806c7","Type":"ContainerStarted","Data":"8d23cf0b96595f63f612ccbb19f69e8525324668aebee7c69e4ac507bf1187dc"} Sep 07 13:04:14.835929 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.835900 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b8ddbf7db-ck7cf" event={"ID":"4c987d10-8899-411d-966d-702b918806c7","Type":"ContainerStarted","Data":"ec7ba81d04f3f4f410d884ef0b0c12cc259ba8205697c76015e1877a42974d0a"} Sep 07 13:04:14.860167 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:14.860107 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b8ddbf7db-ck7cf" podStartSLOduration=1.8600890749999999 podStartE2EDuration="1.860089075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:04:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:04:14.857788467 +0000 UTC m=+215.453293085" watchObservedRunningTime="2026-09-07 13:04:14.860089075 +0000 UTC m=+215.455593715" Sep 07 13:04:24.280114 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:24.280079 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:24.280527 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:24.280128 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:24.284873 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:24.284851 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:04:24.880615 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:04:24.880587 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:05:39.834350 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:05:39.834320 2564 kubelet.go:1648] "Image garbage collection succeeded" Sep 07 13:05:58.936086 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:05:58.936033 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b8ddbf7db-ck7cf"] Sep 07 13:06:23.956132 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:23.956090 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b8ddbf7db-ck7cf" podUID="4c987d10-8899-411d-966d-702b918806c7" containerName="console" containerID="cri-o://8d23cf0b96595f63f612ccbb19f69e8525324668aebee7c69e4ac507bf1187dc" gracePeriod=15 Sep 07 13:06:24.202671 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.202639 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:06:24.307491 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.307454 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-trusted-ca-bundle\") pod \"4c987d10-8899-411d-966d-702b918806c7\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " Sep 07 13:06:24.307702 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.307521 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-console-config\") pod \"4c987d10-8899-411d-966d-702b918806c7\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " Sep 07 13:06:24.307702 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.307559 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6w4x8\" (UniqueName: \"kubernetes.io/projected/4c987d10-8899-411d-966d-702b918806c7-kube-api-access-6w4x8\") pod \"4c987d10-8899-411d-966d-702b918806c7\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " Sep 07 13:06:24.307702 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.307593 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c987d10-8899-411d-966d-702b918806c7-console-oauth-config\") pod \"4c987d10-8899-411d-966d-702b918806c7\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " Sep 07 13:06:24.307702 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.307610 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c987d10-8899-411d-966d-702b918806c7-console-serving-cert\") pod \"4c987d10-8899-411d-966d-702b918806c7\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " Sep 07 13:06:24.307702 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.307627 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-service-ca\") pod \"4c987d10-8899-411d-966d-702b918806c7\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " Sep 07 13:06:24.307702 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.307657 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-oauth-serving-cert\") pod \"4c987d10-8899-411d-966d-702b918806c7\" (UID: \"4c987d10-8899-411d-966d-702b918806c7\") " Sep 07 13:06:24.308020 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.307959 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4c987d10-8899-411d-966d-702b918806c7" (UID: "4c987d10-8899-411d-966d-702b918806c7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:06:24.308020 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.308001 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-console-config" (OuterVolumeSpecName: "console-config") pod "4c987d10-8899-411d-966d-702b918806c7" (UID: "4c987d10-8899-411d-966d-702b918806c7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:06:24.308160 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.308114 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-service-ca" (OuterVolumeSpecName: "service-ca") pod "4c987d10-8899-411d-966d-702b918806c7" (UID: "4c987d10-8899-411d-966d-702b918806c7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:06:24.308263 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.308237 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4c987d10-8899-411d-966d-702b918806c7" (UID: "4c987d10-8899-411d-966d-702b918806c7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:06:24.309876 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.309843 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c987d10-8899-411d-966d-702b918806c7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4c987d10-8899-411d-966d-702b918806c7" (UID: "4c987d10-8899-411d-966d-702b918806c7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:06:24.309876 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.309868 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c987d10-8899-411d-966d-702b918806c7-kube-api-access-6w4x8" (OuterVolumeSpecName: "kube-api-access-6w4x8") pod "4c987d10-8899-411d-966d-702b918806c7" (UID: "4c987d10-8899-411d-966d-702b918806c7"). InnerVolumeSpecName "kube-api-access-6w4x8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:06:24.310019 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.309925 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c987d10-8899-411d-966d-702b918806c7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4c987d10-8899-411d-966d-702b918806c7" (UID: "4c987d10-8899-411d-966d-702b918806c7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:06:24.322493 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.322467 2564 generic.go:356] "Generic (PLEG): container finished" podID="4c987d10-8899-411d-966d-702b918806c7" containerID="8d23cf0b96595f63f612ccbb19f69e8525324668aebee7c69e4ac507bf1187dc" exitCode=0 Sep 07 13:06:24.322588 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.322508 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b8ddbf7db-ck7cf" Sep 07 13:06:24.322588 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.322538 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b8ddbf7db-ck7cf" event={"ID":"4c987d10-8899-411d-966d-702b918806c7","Type":"ContainerDied","Data":"8d23cf0b96595f63f612ccbb19f69e8525324668aebee7c69e4ac507bf1187dc"} Sep 07 13:06:24.322588 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.322576 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b8ddbf7db-ck7cf" event={"ID":"4c987d10-8899-411d-966d-702b918806c7","Type":"ContainerDied","Data":"ec7ba81d04f3f4f410d884ef0b0c12cc259ba8205697c76015e1877a42974d0a"} Sep 07 13:06:24.322719 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.322595 2564 scope.go:117] "RemoveContainer" containerID="8d23cf0b96595f63f612ccbb19f69e8525324668aebee7c69e4ac507bf1187dc" Sep 07 13:06:24.331257 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.331236 2564 scope.go:117] "RemoveContainer" containerID="8d23cf0b96595f63f612ccbb19f69e8525324668aebee7c69e4ac507bf1187dc" Sep 07 13:06:24.331521 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:06:24.331503 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d23cf0b96595f63f612ccbb19f69e8525324668aebee7c69e4ac507bf1187dc\": container with ID starting with 8d23cf0b96595f63f612ccbb19f69e8525324668aebee7c69e4ac507bf1187dc not found: ID does not exist" containerID="8d23cf0b96595f63f612ccbb19f69e8525324668aebee7c69e4ac507bf1187dc" Sep 07 13:06:24.331566 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.331529 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d23cf0b96595f63f612ccbb19f69e8525324668aebee7c69e4ac507bf1187dc"} err="failed to get container status \"8d23cf0b96595f63f612ccbb19f69e8525324668aebee7c69e4ac507bf1187dc\": rpc error: code = NotFound desc = could not find container \"8d23cf0b96595f63f612ccbb19f69e8525324668aebee7c69e4ac507bf1187dc\": container with ID starting with 8d23cf0b96595f63f612ccbb19f69e8525324668aebee7c69e4ac507bf1187dc not found: ID does not exist" Sep 07 13:06:24.347667 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.347631 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b8ddbf7db-ck7cf"] Sep 07 13:06:24.350262 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.350237 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b8ddbf7db-ck7cf"] Sep 07 13:06:24.408794 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.408758 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-console-config\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:06:24.408794 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.408787 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6w4x8\" (UniqueName: \"kubernetes.io/projected/4c987d10-8899-411d-966d-702b918806c7-kube-api-access-6w4x8\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:06:24.408794 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.408798 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c987d10-8899-411d-966d-702b918806c7-console-oauth-config\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:06:24.409028 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.408809 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c987d10-8899-411d-966d-702b918806c7-console-serving-cert\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:06:24.409028 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.408817 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-service-ca\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:06:24.409028 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.408825 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-oauth-serving-cert\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:06:24.409028 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:24.408833 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c987d10-8899-411d-966d-702b918806c7-trusted-ca-bundle\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:06:25.921492 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:06:25.921454 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c987d10-8899-411d-966d-702b918806c7" path="/var/lib/kubelet/pods/4c987d10-8899-411d-966d-702b918806c7/volumes" Sep 07 13:07:48.396688 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:48.396651 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx"] Sep 07 13:07:48.397288 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:48.397109 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c987d10-8899-411d-966d-702b918806c7" containerName="console" Sep 07 13:07:48.397288 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:48.397128 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c987d10-8899-411d-966d-702b918806c7" containerName="console" Sep 07 13:07:48.397288 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:48.397228 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c987d10-8899-411d-966d-702b918806c7" containerName="console" Sep 07 13:07:48.400168 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:48.400147 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx" Sep 07 13:07:48.402561 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:48.402538 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 07 13:07:48.402638 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:48.402540 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 07 13:07:48.403238 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:48.403225 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-7pj8c\"" Sep 07 13:07:48.413144 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:48.413122 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx"] Sep 07 13:07:48.551934 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:48.551897 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxzm6\" (UniqueName: \"kubernetes.io/projected/50f7a4f8-db11-4632-bd06-4e689939d4df-kube-api-access-bxzm6\") pod \"feast-operator-controller-manager-784cdbd879-k5hqx\" (UID: \"50f7a4f8-db11-4632-bd06-4e689939d4df\") " pod="feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx" Sep 07 13:07:48.652783 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:48.652694 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxzm6\" (UniqueName: \"kubernetes.io/projected/50f7a4f8-db11-4632-bd06-4e689939d4df-kube-api-access-bxzm6\") pod \"feast-operator-controller-manager-784cdbd879-k5hqx\" (UID: \"50f7a4f8-db11-4632-bd06-4e689939d4df\") " pod="feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx" Sep 07 13:07:48.661191 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:48.661158 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxzm6\" (UniqueName: \"kubernetes.io/projected/50f7a4f8-db11-4632-bd06-4e689939d4df-kube-api-access-bxzm6\") pod \"feast-operator-controller-manager-784cdbd879-k5hqx\" (UID: \"50f7a4f8-db11-4632-bd06-4e689939d4df\") " pod="feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx" Sep 07 13:07:48.710029 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:48.709993 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx" Sep 07 13:07:48.981766 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:48.981739 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx"] Sep 07 13:07:48.984354 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:07:48.984329 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f7a4f8_db11_4632_bd06_4e689939d4df.slice/crio-5de7dfa526954cc78f9b47ead53cb1c32e0cde4d843b277a2266282f0aa0c995 WatchSource:0}: Error finding container 5de7dfa526954cc78f9b47ead53cb1c32e0cde4d843b277a2266282f0aa0c995: Status 404 returned error can't find the container with id 5de7dfa526954cc78f9b47ead53cb1c32e0cde4d843b277a2266282f0aa0c995 Sep 07 13:07:48.986168 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:48.986147 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 13:07:49.647678 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:49.647640 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx" event={"ID":"50f7a4f8-db11-4632-bd06-4e689939d4df","Type":"ContainerStarted","Data":"5de7dfa526954cc78f9b47ead53cb1c32e0cde4d843b277a2266282f0aa0c995"} Sep 07 13:07:52.660677 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:52.660638 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx" event={"ID":"50f7a4f8-db11-4632-bd06-4e689939d4df","Type":"ContainerStarted","Data":"c0f8ffe6ab8dafa16e35f4d0d5efabb0ff9f67bcef879661ca96cae586ca4d7c"} Sep 07 13:07:52.661103 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:52.660786 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx" Sep 07 13:07:52.678520 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:07:52.678475 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx" podStartSLOduration=2.02665125 podStartE2EDuration="4.678460957s" totalImagesPullingTime="2.651809707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:07:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:07:52.677390019 +0000 UTC m=+433.272894634" watchObservedRunningTime="2026-09-07 13:07:52.678460957 +0000 UTC m=+433.273965575" Sep 07 13:08:03.667127 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:03.667040 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx" Sep 07 13:08:20.288282 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.288240 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d99c97c9f-5p8gn"] Sep 07 13:08:20.291470 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.291448 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.293763 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.293736 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jwmrq\"" Sep 07 13:08:20.293888 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.293736 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 07 13:08:20.293888 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.293743 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 07 13:08:20.294588 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.294554 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 07 13:08:20.294714 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.294600 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 07 13:08:20.294821 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.294807 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 07 13:08:20.298533 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.298512 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 07 13:08:20.303456 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.303433 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d99c97c9f-5p8gn"] Sep 07 13:08:20.389426 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.389382 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e29cc86-2a68-4f98-93b8-f8ee7296966a-trusted-ca-bundle\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.389426 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.389423 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwb6l\" (UniqueName: \"kubernetes.io/projected/2e29cc86-2a68-4f98-93b8-f8ee7296966a-kube-api-access-nwb6l\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.389662 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.389444 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e29cc86-2a68-4f98-93b8-f8ee7296966a-console-serving-cert\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.389662 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.389518 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e29cc86-2a68-4f98-93b8-f8ee7296966a-console-oauth-config\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.389662 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.389548 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e29cc86-2a68-4f98-93b8-f8ee7296966a-oauth-serving-cert\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.389662 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.389573 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e29cc86-2a68-4f98-93b8-f8ee7296966a-service-ca\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.389662 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.389623 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e29cc86-2a68-4f98-93b8-f8ee7296966a-console-config\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.490714 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.490660 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e29cc86-2a68-4f98-93b8-f8ee7296966a-service-ca\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.490714 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.490725 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e29cc86-2a68-4f98-93b8-f8ee7296966a-console-config\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.490930 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.490753 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e29cc86-2a68-4f98-93b8-f8ee7296966a-trusted-ca-bundle\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.490930 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.490773 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwb6l\" (UniqueName: \"kubernetes.io/projected/2e29cc86-2a68-4f98-93b8-f8ee7296966a-kube-api-access-nwb6l\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.490930 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.490792 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e29cc86-2a68-4f98-93b8-f8ee7296966a-console-serving-cert\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.491046 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.490936 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e29cc86-2a68-4f98-93b8-f8ee7296966a-console-oauth-config\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.491046 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.490981 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e29cc86-2a68-4f98-93b8-f8ee7296966a-oauth-serving-cert\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.491548 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.491521 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e29cc86-2a68-4f98-93b8-f8ee7296966a-service-ca\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.491686 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.491546 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e29cc86-2a68-4f98-93b8-f8ee7296966a-console-config\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.491686 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.491609 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e29cc86-2a68-4f98-93b8-f8ee7296966a-oauth-serving-cert\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.491812 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.491792 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e29cc86-2a68-4f98-93b8-f8ee7296966a-trusted-ca-bundle\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.493284 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.493265 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e29cc86-2a68-4f98-93b8-f8ee7296966a-console-serving-cert\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.493400 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.493382 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e29cc86-2a68-4f98-93b8-f8ee7296966a-console-oauth-config\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.500290 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.500271 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwb6l\" (UniqueName: \"kubernetes.io/projected/2e29cc86-2a68-4f98-93b8-f8ee7296966a-kube-api-access-nwb6l\") pod \"console-5d99c97c9f-5p8gn\" (UID: \"2e29cc86-2a68-4f98-93b8-f8ee7296966a\") " pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.601452 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.601364 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:20.875931 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:20.875906 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d99c97c9f-5p8gn"] Sep 07 13:08:20.878161 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:08:20.878129 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e29cc86_2a68_4f98_93b8_f8ee7296966a.slice/crio-464f3d2706bfc00319d4f45d2c7dccb38208b8ae72cf7afe79f3cf3b8d0755bb WatchSource:0}: Error finding container 464f3d2706bfc00319d4f45d2c7dccb38208b8ae72cf7afe79f3cf3b8d0755bb: Status 404 returned error can't find the container with id 464f3d2706bfc00319d4f45d2c7dccb38208b8ae72cf7afe79f3cf3b8d0755bb Sep 07 13:08:21.778548 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:21.778509 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d99c97c9f-5p8gn" event={"ID":"2e29cc86-2a68-4f98-93b8-f8ee7296966a","Type":"ContainerStarted","Data":"d9021f8476f7f66f53352fc57d27540cc5162398e5015254021fbaf7ccdac989"} Sep 07 13:08:21.778920 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:21.778571 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d99c97c9f-5p8gn" event={"ID":"2e29cc86-2a68-4f98-93b8-f8ee7296966a","Type":"ContainerStarted","Data":"464f3d2706bfc00319d4f45d2c7dccb38208b8ae72cf7afe79f3cf3b8d0755bb"} Sep 07 13:08:21.798677 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:21.798624 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d99c97c9f-5p8gn" podStartSLOduration=1.798610767 podStartE2EDuration="1.798610767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:08:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:08:21.797314266 +0000 UTC m=+462.392818882" watchObservedRunningTime="2026-09-07 13:08:21.798610767 +0000 UTC m=+462.394115382" Sep 07 13:08:28.718096 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.718064 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f"] Sep 07 13:08:28.725987 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.725966 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.728424 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.728395 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 07 13:08:28.728535 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.728435 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 07 13:08:28.728535 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.728479 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 07 13:08:28.728734 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.728716 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 07 13:08:28.728818 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.728742 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 07 13:08:28.728818 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.728761 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 07 13:08:28.728818 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.728787 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 07 13:08:28.728972 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.728819 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-cx2vp\"" Sep 07 13:08:28.735482 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.735459 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f"] Sep 07 13:08:28.864763 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.864725 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-offline-tls\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.864958 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.864784 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksjcq\" (UniqueName: \"kubernetes.io/projected/146cf6fd-8d65-4368-bac0-621b1d35dc49-kube-api-access-ksjcq\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.864958 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.864817 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-registry-tls\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.864958 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.864847 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-online-tls\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.864958 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.864867 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/146cf6fd-8d65-4368-bac0-621b1d35dc49-feast-data\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.864958 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.864905 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-ui-tls\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.965842 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.965802 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksjcq\" (UniqueName: \"kubernetes.io/projected/146cf6fd-8d65-4368-bac0-621b1d35dc49-kube-api-access-ksjcq\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.965842 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.965845 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-registry-tls\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.966045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.965870 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-online-tls\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.966131 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.966088 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/146cf6fd-8d65-4368-bac0-621b1d35dc49-feast-data\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.966189 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.966145 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-ui-tls\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.966256 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.966242 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-offline-tls\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.966347 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:08:28.966325 2564 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 07 13:08:28.966392 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.966374 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/146cf6fd-8d65-4368-bac0-621b1d35dc49-feast-data\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.966442 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:08:28.966420 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-offline-tls podName:146cf6fd-8d65-4368-bac0-621b1d35dc49 nodeName:}" failed. No retries permitted until 2026-09-07 13:08:29.46639078 +0000 UTC m=+470.061895393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-offline-tls") pod "feast-simple-feast-setup-77b88b9d59-nkt7f" (UID: "146cf6fd-8d65-4368-bac0-621b1d35dc49") : secret "feast-simple-feast-setup-offline-tls" not found Sep 07 13:08:28.973603 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.973528 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-registry-tls\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.973736 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.973681 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-online-tls\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.974090 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.974072 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-ui-tls\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:28.975375 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:28.975354 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksjcq\" (UniqueName: \"kubernetes.io/projected/146cf6fd-8d65-4368-bac0-621b1d35dc49-kube-api-access-ksjcq\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:29.472168 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:29.472127 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-offline-tls\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:29.474428 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:29.474398 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-offline-tls\") pod \"feast-simple-feast-setup-77b88b9d59-nkt7f\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:29.650832 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:29.650791 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:08:29.934604 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:08:29.934570 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod146cf6fd_8d65_4368_bac0_621b1d35dc49.slice/crio-7e4473da64e449fe933060e79e6f0a1bfd3921369fd51d1bd19c1da1983cee4f WatchSource:0}: Error finding container 7e4473da64e449fe933060e79e6f0a1bfd3921369fd51d1bd19c1da1983cee4f: Status 404 returned error can't find the container with id 7e4473da64e449fe933060e79e6f0a1bfd3921369fd51d1bd19c1da1983cee4f Sep 07 13:08:29.934995 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:29.934669 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f"] Sep 07 13:08:30.602373 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:30.602337 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:30.602586 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:30.602399 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:30.608465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:30.608438 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:30.819993 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:30.819947 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" event={"ID":"146cf6fd-8d65-4368-bac0-621b1d35dc49","Type":"ContainerStarted","Data":"7e4473da64e449fe933060e79e6f0a1bfd3921369fd51d1bd19c1da1983cee4f"} Sep 07 13:08:30.825338 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:30.825310 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d99c97c9f-5p8gn" Sep 07 13:08:55.939796 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:08:55.939758 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" event={"ID":"146cf6fd-8d65-4368-bac0-621b1d35dc49","Type":"ContainerStarted","Data":"6e0baf1f15f5108ccb3bb103520af15dc28f332ea11f0d540f8df2b080779872"} Sep 07 13:09:00.964969 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:00.964938 2564 generic.go:356] "Generic (PLEG): container finished" podID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerID="6e0baf1f15f5108ccb3bb103520af15dc28f332ea11f0d540f8df2b080779872" exitCode=0 Sep 07 13:09:00.965350 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:00.965012 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" event={"ID":"146cf6fd-8d65-4368-bac0-621b1d35dc49","Type":"ContainerDied","Data":"6e0baf1f15f5108ccb3bb103520af15dc28f332ea11f0d540f8df2b080779872"} Sep 07 13:09:01.970481 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:01.970437 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" event={"ID":"146cf6fd-8d65-4368-bac0-621b1d35dc49","Type":"ContainerStarted","Data":"e85d22ecd33dafc361c4d42878901d5b4efb223d34051e93cd19c778a1a89ac0"} Sep 07 13:09:06.992427 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:06.992392 2564 generic.go:356] "Generic (PLEG): container finished" podID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerID="e85d22ecd33dafc361c4d42878901d5b4efb223d34051e93cd19c778a1a89ac0" exitCode=0 Sep 07 13:09:06.992427 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:06.992430 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" event={"ID":"146cf6fd-8d65-4368-bac0-621b1d35dc49","Type":"ContainerDied","Data":"e85d22ecd33dafc361c4d42878901d5b4efb223d34051e93cd19c778a1a89ac0"} Sep 07 13:09:07.002527 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:07.002503 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 07 13:09:08.003462 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:08.003416 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" event={"ID":"146cf6fd-8d65-4368-bac0-621b1d35dc49","Type":"ContainerStarted","Data":"27513986d9415b77341a6ab9e1b4d3857289b3bf72a46aef1a8908c577e9ff58"} Sep 07 13:09:08.003964 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:08.003487 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" event={"ID":"146cf6fd-8d65-4368-bac0-621b1d35dc49","Type":"ContainerStarted","Data":"d06c0dce7289c8886240354ec34bce1c4628d65d2089690d77b45e897798c66a"} Sep 07 13:09:08.003964 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:08.003510 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" event={"ID":"146cf6fd-8d65-4368-bac0-621b1d35dc49","Type":"ContainerStarted","Data":"e9a8fe15c6abdb85b0ac85e89d0f7cd743002b5e8b28daf76e22cd3143454c13"} Sep 07 13:09:08.003964 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:08.003549 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" event={"ID":"146cf6fd-8d65-4368-bac0-621b1d35dc49","Type":"ContainerStarted","Data":"e369b410f996a6427895f15c04a709ca25a32486a01321fcb44ba4109044614d"} Sep 07 13:09:08.032752 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:08.032670 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podStartSLOduration=14.554122628 podStartE2EDuration="40.032646102s" totalImagesPullingTime="25.478523474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:09:08.032137342 +0000 UTC m=+508.627641982" watchObservedRunningTime="2026-09-07 13:09:08.032646102 +0000 UTC m=+508.628150720" Sep 07 13:09:08.652188 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:08.652144 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:09:08.652401 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:08.652209 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:09:08.652401 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:08.652230 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:09:08.652401 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:08.652247 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:09:08.654376 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:08.654339 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused" Sep 07 13:09:08.654518 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:08.654373 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.26:8816: connect: connection refused" Sep 07 13:09:08.654518 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:08.654464 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.26:6571: connect: connection refused" Sep 07 13:09:08.654639 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:08.654566 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Sep 07 13:09:09.651188 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:09.651139 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:09:09.651686 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:09.651204 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:09:09.651686 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:09.651231 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:09:09.651686 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:09.651286 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:09:11.651888 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:11.651827 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.26:8816: connect: connection refused" Sep 07 13:09:11.652444 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:11.651827 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.26:6571: connect: connection refused" Sep 07 13:09:11.652444 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:11.651839 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Sep 07 13:09:11.652660 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:11.652626 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused" Sep 07 13:09:14.651648 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:14.651599 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.26:8816: connect: connection refused" Sep 07 13:09:14.652270 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:14.651599 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.26:6571: connect: connection refused" Sep 07 13:09:14.652421 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:14.651605 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Sep 07 13:09:14.652421 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:14.651613 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused" Sep 07 13:09:17.652210 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:17.652177 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:09:17.652714 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:17.652350 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:09:17.652714 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:17.652613 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:09:17.653150 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:17.653125 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:09:17.653252 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:17.653186 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:09:17.653252 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:17.653223 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:09:17.657428 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:17.657406 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:09:18.066778 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:09:18.066744 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:10:20.975884 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:20.975846 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f"] Sep 07 13:10:20.976415 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:20.976148 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="registry" containerID="cri-o://e369b410f996a6427895f15c04a709ca25a32486a01321fcb44ba4109044614d" gracePeriod=30 Sep 07 13:10:20.976415 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:20.976267 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="offline" containerID="cri-o://d06c0dce7289c8886240354ec34bce1c4628d65d2089690d77b45e897798c66a" gracePeriod=30 Sep 07 13:10:20.976415 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:20.976292 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="ui" containerID="cri-o://27513986d9415b77341a6ab9e1b4d3857289b3bf72a46aef1a8908c577e9ff58" gracePeriod=30 Sep 07 13:10:20.976415 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:20.976290 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="online" containerID="cri-o://e9a8fe15c6abdb85b0ac85e89d0f7cd743002b5e8b28daf76e22cd3143454c13" gracePeriod=30 Sep 07 13:10:21.416855 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.416812 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr"] Sep 07 13:10:21.420880 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.420857 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.423430 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.423405 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-l9zmk\"" Sep 07 13:10:21.436969 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.436928 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr"] Sep 07 13:10:21.522047 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.522009 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-online-tls\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.522267 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.522068 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpcmw\" (UniqueName: \"kubernetes.io/projected/1712af5d-d5a3-4841-92cb-72109f23e22d-kube-api-access-qpcmw\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.522267 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.522160 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-offline-tls\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.522267 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.522193 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1712af5d-d5a3-4841-92cb-72109f23e22d-feast-data\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.522267 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.522212 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-ui-tls\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.522486 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.522296 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-registry-tls\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.623508 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.623450 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-online-tls\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.623508 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.623510 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpcmw\" (UniqueName: \"kubernetes.io/projected/1712af5d-d5a3-4841-92cb-72109f23e22d-kube-api-access-qpcmw\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.623764 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.623563 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-offline-tls\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.623764 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.623588 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1712af5d-d5a3-4841-92cb-72109f23e22d-feast-data\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.623764 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.623612 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-ui-tls\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.623764 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.623641 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-registry-tls\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.624034 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.623999 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1712af5d-d5a3-4841-92cb-72109f23e22d-feast-data\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.626365 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.626329 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-ui-tls\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.626806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.626782 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-offline-tls\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.626855 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.626785 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-online-tls\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.626855 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.626839 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-registry-tls\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.633806 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.633777 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpcmw\" (UniqueName: \"kubernetes.io/projected/1712af5d-d5a3-4841-92cb-72109f23e22d-kube-api-access-qpcmw\") pod \"feast-simple-feast-setup-77b88b9d59-rhvkr\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:21.732677 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:21.732627 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:22.031379 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:22.031350 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr"] Sep 07 13:10:22.033768 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:10:22.033740 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1712af5d_d5a3_4841_92cb_72109f23e22d.slice/crio-afd4cd5617cc941f4ca5632ea1a45786b787bea7cc3b0b4b68d4aa4d359db7a6 WatchSource:0}: Error finding container afd4cd5617cc941f4ca5632ea1a45786b787bea7cc3b0b4b68d4aa4d359db7a6: Status 404 returned error can't find the container with id afd4cd5617cc941f4ca5632ea1a45786b787bea7cc3b0b4b68d4aa4d359db7a6 Sep 07 13:10:22.324273 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:22.324172 2564 generic.go:356] "Generic (PLEG): container finished" podID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerID="27513986d9415b77341a6ab9e1b4d3857289b3bf72a46aef1a8908c577e9ff58" exitCode=0 Sep 07 13:10:22.324273 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:22.324212 2564 generic.go:356] "Generic (PLEG): container finished" podID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerID="e9a8fe15c6abdb85b0ac85e89d0f7cd743002b5e8b28daf76e22cd3143454c13" exitCode=0 Sep 07 13:10:22.324505 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:22.324306 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" event={"ID":"146cf6fd-8d65-4368-bac0-621b1d35dc49","Type":"ContainerDied","Data":"27513986d9415b77341a6ab9e1b4d3857289b3bf72a46aef1a8908c577e9ff58"} Sep 07 13:10:22.324505 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:22.324349 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" event={"ID":"146cf6fd-8d65-4368-bac0-621b1d35dc49","Type":"ContainerDied","Data":"e9a8fe15c6abdb85b0ac85e89d0f7cd743002b5e8b28daf76e22cd3143454c13"} Sep 07 13:10:22.326445 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:22.326406 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" event={"ID":"1712af5d-d5a3-4841-92cb-72109f23e22d","Type":"ContainerStarted","Data":"55f471d340a45f1315b5e9776f6d7dc138bfef8ce6df4ea51a8cf7f3f8eb8e72"} Sep 07 13:10:22.326507 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:22.326468 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" event={"ID":"1712af5d-d5a3-4841-92cb-72109f23e22d","Type":"ContainerStarted","Data":"afd4cd5617cc941f4ca5632ea1a45786b787bea7cc3b0b4b68d4aa4d359db7a6"} Sep 07 13:10:27.349481 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:27.349446 2564 generic.go:356] "Generic (PLEG): container finished" podID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerID="55f471d340a45f1315b5e9776f6d7dc138bfef8ce6df4ea51a8cf7f3f8eb8e72" exitCode=0 Sep 07 13:10:27.349862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:27.349522 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" event={"ID":"1712af5d-d5a3-4841-92cb-72109f23e22d","Type":"ContainerDied","Data":"55f471d340a45f1315b5e9776f6d7dc138bfef8ce6df4ea51a8cf7f3f8eb8e72"} Sep 07 13:10:27.653011 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:27.652950 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused" Sep 07 13:10:28.064026 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:28.063972 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Sep 07 13:10:28.355707 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:28.355615 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" event={"ID":"1712af5d-d5a3-4841-92cb-72109f23e22d","Type":"ContainerStarted","Data":"a255c92cf984c20a19194a01c4ecca02180e10d338c8e215c66c8178d3ed9218"} Sep 07 13:10:34.390295 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:34.390255 2564 generic.go:356] "Generic (PLEG): container finished" podID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerID="a255c92cf984c20a19194a01c4ecca02180e10d338c8e215c66c8178d3ed9218" exitCode=0 Sep 07 13:10:34.390677 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:34.390328 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" event={"ID":"1712af5d-d5a3-4841-92cb-72109f23e22d","Type":"ContainerDied","Data":"a255c92cf984c20a19194a01c4ecca02180e10d338c8e215c66c8178d3ed9218"} Sep 07 13:10:35.401270 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:35.401226 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" event={"ID":"1712af5d-d5a3-4841-92cb-72109f23e22d","Type":"ContainerStarted","Data":"557f6f4ba3114a8273812d41a3899476387926fc0bc21920d3d7b7b1173b2d87"} Sep 07 13:10:35.401770 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:35.401291 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" event={"ID":"1712af5d-d5a3-4841-92cb-72109f23e22d","Type":"ContainerStarted","Data":"fa2b54e295a66c3f9a5ffca69abbf74f25b69c2fac518e2aed1ba06da981c012"} Sep 07 13:10:35.401770 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:35.401311 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" event={"ID":"1712af5d-d5a3-4841-92cb-72109f23e22d","Type":"ContainerStarted","Data":"2492435944a8dd37b8ab9fa788a842729653f5e51309c3e1f872b80d9ca822a4"} Sep 07 13:10:35.401770 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:35.401329 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" event={"ID":"1712af5d-d5a3-4841-92cb-72109f23e22d","Type":"ContainerStarted","Data":"c2c42b20379956bd27bb708f9ea3fc9f92486e856bb981c3ca2bf986280e1a38"} Sep 07 13:10:35.433240 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:35.433164 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podStartSLOduration=14.433131781 podStartE2EDuration="14.433131781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:10:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:10:35.433135013 +0000 UTC m=+596.028639630" watchObservedRunningTime="2026-09-07 13:10:35.433131781 +0000 UTC m=+596.028636397" Sep 07 13:10:36.733120 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:36.733073 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:36.733734 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:36.733699 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:36.733734 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:36.733734 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:36.733918 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:36.733753 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:36.735877 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:36.735667 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 07 13:10:36.735877 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:36.735683 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.27:8816: connect: connection refused" Sep 07 13:10:36.735877 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:36.735709 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.27:6571: connect: connection refused" Sep 07 13:10:36.735877 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:36.735815 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Sep 07 13:10:37.653961 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:37.653908 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused" Sep 07 13:10:38.063943 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:38.063886 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Sep 07 13:10:39.733477 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:39.733421 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.27:6571: connect: connection refused" Sep 07 13:10:39.733992 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:39.733430 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Sep 07 13:10:39.733992 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:39.733433 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 07 13:10:39.733992 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:39.733869 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.27:8816: connect: connection refused" Sep 07 13:10:41.733791 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:41.733734 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:41.734326 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:41.733814 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:41.734326 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:41.733838 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:41.734326 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:41.733855 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:42.733313 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:42.733249 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 07 13:10:42.734180 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:42.734153 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:42.734532 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:42.734269 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:42.735151 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:42.735131 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:42.735246 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:42.735185 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:42.739878 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:42.739856 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:43.455255 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:43.455216 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:45.733745 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:45.733714 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:46.464225 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:46.464194 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:10:47.653169 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:47.653110 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused" Sep 07 13:10:47.653627 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:47.653270 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:10:48.064365 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:48.064317 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Sep 07 13:10:48.064571 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:48.064434 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:10:51.486554 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.486522 2564 generic.go:356] "Generic (PLEG): container finished" podID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerID="d06c0dce7289c8886240354ec34bce1c4628d65d2089690d77b45e897798c66a" exitCode=137 Sep 07 13:10:51.486554 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.486547 2564 generic.go:356] "Generic (PLEG): container finished" podID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerID="e369b410f996a6427895f15c04a709ca25a32486a01321fcb44ba4109044614d" exitCode=137 Sep 07 13:10:51.487007 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.486595 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" event={"ID":"146cf6fd-8d65-4368-bac0-621b1d35dc49","Type":"ContainerDied","Data":"d06c0dce7289c8886240354ec34bce1c4628d65d2089690d77b45e897798c66a"} Sep 07 13:10:51.487007 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.486637 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" event={"ID":"146cf6fd-8d65-4368-bac0-621b1d35dc49","Type":"ContainerDied","Data":"e369b410f996a6427895f15c04a709ca25a32486a01321fcb44ba4109044614d"} Sep 07 13:10:51.625074 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.625025 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:10:51.802609 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.802522 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-registry-tls\") pod \"146cf6fd-8d65-4368-bac0-621b1d35dc49\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " Sep 07 13:10:51.802609 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.802568 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-online-tls\") pod \"146cf6fd-8d65-4368-bac0-621b1d35dc49\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " Sep 07 13:10:51.802609 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.802594 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/146cf6fd-8d65-4368-bac0-621b1d35dc49-feast-data\") pod \"146cf6fd-8d65-4368-bac0-621b1d35dc49\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " Sep 07 13:10:51.802836 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.802732 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksjcq\" (UniqueName: \"kubernetes.io/projected/146cf6fd-8d65-4368-bac0-621b1d35dc49-kube-api-access-ksjcq\") pod \"146cf6fd-8d65-4368-bac0-621b1d35dc49\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " Sep 07 13:10:51.802836 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.802783 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-offline-tls\") pod \"146cf6fd-8d65-4368-bac0-621b1d35dc49\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " Sep 07 13:10:51.802946 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.802928 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-ui-tls\") pod \"146cf6fd-8d65-4368-bac0-621b1d35dc49\" (UID: \"146cf6fd-8d65-4368-bac0-621b1d35dc49\") " Sep 07 13:10:51.803222 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.803192 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/146cf6fd-8d65-4368-bac0-621b1d35dc49-feast-data" (OuterVolumeSpecName: "feast-data") pod "146cf6fd-8d65-4368-bac0-621b1d35dc49" (UID: "146cf6fd-8d65-4368-bac0-621b1d35dc49"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:10:51.805040 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.805013 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-online-tls" (OuterVolumeSpecName: "online-tls") pod "146cf6fd-8d65-4368-bac0-621b1d35dc49" (UID: "146cf6fd-8d65-4368-bac0-621b1d35dc49"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:10:51.805040 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.805030 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "146cf6fd-8d65-4368-bac0-621b1d35dc49" (UID: "146cf6fd-8d65-4368-bac0-621b1d35dc49"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:10:51.805259 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.805078 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "146cf6fd-8d65-4368-bac0-621b1d35dc49" (UID: "146cf6fd-8d65-4368-bac0-621b1d35dc49"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:10:51.805259 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.805230 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/146cf6fd-8d65-4368-bac0-621b1d35dc49-kube-api-access-ksjcq" (OuterVolumeSpecName: "kube-api-access-ksjcq") pod "146cf6fd-8d65-4368-bac0-621b1d35dc49" (UID: "146cf6fd-8d65-4368-bac0-621b1d35dc49"). InnerVolumeSpecName "kube-api-access-ksjcq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:10:51.805259 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.805233 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "146cf6fd-8d65-4368-bac0-621b1d35dc49" (UID: "146cf6fd-8d65-4368-bac0-621b1d35dc49"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:10:51.903932 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.903876 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksjcq\" (UniqueName: \"kubernetes.io/projected/146cf6fd-8d65-4368-bac0-621b1d35dc49-kube-api-access-ksjcq\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:10:51.903932 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.903930 2564 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-offline-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:10:51.903932 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.903945 2564 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-ui-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:10:51.903932 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.903955 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-registry-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:10:51.904260 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.903993 2564 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/146cf6fd-8d65-4368-bac0-621b1d35dc49-online-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:10:51.904260 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:51.904004 2564 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/146cf6fd-8d65-4368-bac0-621b1d35dc49-feast-data\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:10:52.494354 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:52.494324 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" Sep 07 13:10:52.494805 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:52.494320 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f" event={"ID":"146cf6fd-8d65-4368-bac0-621b1d35dc49","Type":"ContainerDied","Data":"7e4473da64e449fe933060e79e6f0a1bfd3921369fd51d1bd19c1da1983cee4f"} Sep 07 13:10:52.494805 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:52.494458 2564 scope.go:117] "RemoveContainer" containerID="27513986d9415b77341a6ab9e1b4d3857289b3bf72a46aef1a8908c577e9ff58" Sep 07 13:10:52.503044 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:52.503021 2564 scope.go:117] "RemoveContainer" containerID="d06c0dce7289c8886240354ec34bce1c4628d65d2089690d77b45e897798c66a" Sep 07 13:10:52.510826 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:52.510804 2564 scope.go:117] "RemoveContainer" containerID="e9a8fe15c6abdb85b0ac85e89d0f7cd743002b5e8b28daf76e22cd3143454c13" Sep 07 13:10:52.515757 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:52.515728 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f"] Sep 07 13:10:52.520048 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:52.520014 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-77b88b9d59-nkt7f"] Sep 07 13:10:52.520285 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:52.520268 2564 scope.go:117] "RemoveContainer" containerID="e369b410f996a6427895f15c04a709ca25a32486a01321fcb44ba4109044614d" Sep 07 13:10:52.527933 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:52.527916 2564 scope.go:117] "RemoveContainer" containerID="e85d22ecd33dafc361c4d42878901d5b4efb223d34051e93cd19c778a1a89ac0" Sep 07 13:10:52.535257 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:52.535231 2564 scope.go:117] "RemoveContainer" containerID="6e0baf1f15f5108ccb3bb103520af15dc28f332ea11f0d540f8df2b080779872" Sep 07 13:10:53.922309 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:10:53.922273 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" path="/var/lib/kubelet/pods/146cf6fd-8d65-4368-bac0-621b1d35dc49/volumes" Sep 07 13:11:48.812884 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.812845 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b"] Sep 07 13:11:48.813335 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.813181 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="feast-apply" Sep 07 13:11:48.813335 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.813194 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="feast-apply" Sep 07 13:11:48.813335 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.813204 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="ui" Sep 07 13:11:48.813335 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.813210 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="ui" Sep 07 13:11:48.813335 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.813218 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="registry" Sep 07 13:11:48.813335 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.813223 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="registry" Sep 07 13:11:48.813335 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.813238 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="online" Sep 07 13:11:48.813335 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.813242 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="online" Sep 07 13:11:48.813335 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.813249 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="offline" Sep 07 13:11:48.813335 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.813254 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="offline" Sep 07 13:11:48.813335 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.813263 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="feast-init" Sep 07 13:11:48.813335 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.813268 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="feast-init" Sep 07 13:11:48.813335 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.813331 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="offline" Sep 07 13:11:48.813335 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.813340 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="online" Sep 07 13:11:48.813744 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.813350 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="registry" Sep 07 13:11:48.813744 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.813357 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="146cf6fd-8d65-4368-bac0-621b1d35dc49" containerName="ui" Sep 07 13:11:48.817878 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.817855 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.822782 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.822760 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 07 13:11:48.822935 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.822916 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 07 13:11:48.823687 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.823660 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 07 13:11:48.823824 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.823802 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 07 13:11:48.823943 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.823806 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 07 13:11:48.823943 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.823875 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-vwfkd\"" Sep 07 13:11:48.824079 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.823977 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 07 13:11:48.825777 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.825760 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 07 13:11:48.866339 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.866304 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b"] Sep 07 13:11:48.869987 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.869959 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-offline-tls\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.870127 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.870031 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-ui-tls\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.870127 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.870102 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96sgp\" (UniqueName: \"kubernetes.io/projected/af1211af-50e7-4a82-8da9-b703a5a3af99-kube-api-access-96sgp\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.870223 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.870145 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-online-tls\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.870223 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.870165 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/af1211af-50e7-4a82-8da9-b703a5a3af99-feast-data\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.870223 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.870219 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/af1211af-50e7-4a82-8da9-b703a5a3af99-registry-tls\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.970600 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.970554 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-offline-tls\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.970817 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.970635 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-ui-tls\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.970817 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.970663 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96sgp\" (UniqueName: \"kubernetes.io/projected/af1211af-50e7-4a82-8da9-b703a5a3af99-kube-api-access-96sgp\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.970817 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.970697 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-online-tls\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.970817 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.970720 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/af1211af-50e7-4a82-8da9-b703a5a3af99-feast-data\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.970817 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.970764 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/af1211af-50e7-4a82-8da9-b703a5a3af99-registry-tls\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.971216 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.971184 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/af1211af-50e7-4a82-8da9-b703a5a3af99-feast-data\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.971606 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.971578 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/af1211af-50e7-4a82-8da9-b703a5a3af99-registry-tls\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.973215 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.973196 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-online-tls\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.973337 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.973318 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-offline-tls\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.973337 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.973330 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-ui-tls\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:48.984208 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:48.984180 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96sgp\" (UniqueName: \"kubernetes.io/projected/af1211af-50e7-4a82-8da9-b703a5a3af99-kube-api-access-96sgp\") pod \"feast-simple-feast-remote-setup-68df65b795-vsc5b\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:49.127210 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:49.127122 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:11:49.421119 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:49.421092 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b"] Sep 07 13:11:49.422696 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:11:49.422663 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf1211af_50e7_4a82_8da9_b703a5a3af99.slice/crio-8f79faca227e49c4648be5d10806ded825b5d1a932b380ee309ff634dfdc4d4c WatchSource:0}: Error finding container 8f79faca227e49c4648be5d10806ded825b5d1a932b380ee309ff634dfdc4d4c: Status 404 returned error can't find the container with id 8f79faca227e49c4648be5d10806ded825b5d1a932b380ee309ff634dfdc4d4c Sep 07 13:11:49.730020 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:49.729979 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" event={"ID":"af1211af-50e7-4a82-8da9-b703a5a3af99","Type":"ContainerStarted","Data":"d898833f66cdb5c6834d63c7ee4415873ec93eac923130232c755a986c15a1f2"} Sep 07 13:11:49.730215 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:49.730046 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" event={"ID":"af1211af-50e7-4a82-8da9-b703a5a3af99","Type":"ContainerStarted","Data":"8f79faca227e49c4648be5d10806ded825b5d1a932b380ee309ff634dfdc4d4c"} Sep 07 13:11:54.752709 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:54.752673 2564 generic.go:356] "Generic (PLEG): container finished" podID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerID="d898833f66cdb5c6834d63c7ee4415873ec93eac923130232c755a986c15a1f2" exitCode=0 Sep 07 13:11:54.753104 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:54.752744 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" event={"ID":"af1211af-50e7-4a82-8da9-b703a5a3af99","Type":"ContainerDied","Data":"d898833f66cdb5c6834d63c7ee4415873ec93eac923130232c755a986c15a1f2"} Sep 07 13:11:55.759302 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:11:55.759266 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" event={"ID":"af1211af-50e7-4a82-8da9-b703a5a3af99","Type":"ContainerStarted","Data":"6dd9fd99bc2b1ef32820ed8d8f4029bcf75abc9eb5b8a7ebd2fa3251a329bcee"} Sep 07 13:12:22.876074 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:22.876015 2564 generic.go:356] "Generic (PLEG): container finished" podID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerID="6dd9fd99bc2b1ef32820ed8d8f4029bcf75abc9eb5b8a7ebd2fa3251a329bcee" exitCode=0 Sep 07 13:12:22.876592 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:22.876092 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" event={"ID":"af1211af-50e7-4a82-8da9-b703a5a3af99","Type":"ContainerDied","Data":"6dd9fd99bc2b1ef32820ed8d8f4029bcf75abc9eb5b8a7ebd2fa3251a329bcee"} Sep 07 13:12:23.885758 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:23.885725 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" event={"ID":"af1211af-50e7-4a82-8da9-b703a5a3af99","Type":"ContainerStarted","Data":"830cf561186832fae7420e92aaf7ecb5d2b68eee47a871ce5341b27b993d4295"} Sep 07 13:12:23.886374 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:23.885773 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" event={"ID":"af1211af-50e7-4a82-8da9-b703a5a3af99","Type":"ContainerStarted","Data":"437130537c93ae5da5a8924d0f0f0af41559cdadcdf3a79958b6dbc823dfcc05"} Sep 07 13:12:23.886374 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:23.885787 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" event={"ID":"af1211af-50e7-4a82-8da9-b703a5a3af99","Type":"ContainerStarted","Data":"e8fd977ed9a84b1081110c8177835e295bfae81a4692582147c8f6f5b576eef8"} Sep 07 13:12:23.912115 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:23.912047 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podStartSLOduration=35.912030961 podStartE2EDuration="35.912030961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:11:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:12:23.910934784 +0000 UTC m=+704.506439412" watchObservedRunningTime="2026-09-07 13:12:23.912030961 +0000 UTC m=+704.507535576" Sep 07 13:12:25.127823 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:25.127749 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:12:25.127823 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:25.127834 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:12:25.128406 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:25.127855 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:12:25.129270 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:25.129234 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 07 13:12:25.129401 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:25.129274 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 07 13:12:25.129401 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:25.129274 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.28:8816: connect: connection refused" Sep 07 13:12:28.128478 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:28.128429 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.28:8816: connect: connection refused" Sep 07 13:12:28.128994 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:28.128430 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 07 13:12:28.128994 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:28.128448 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 07 13:12:29.127826 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:29.127780 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:12:29.128011 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:29.127845 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:12:29.128011 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:29.127866 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:12:31.127753 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:31.127696 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 07 13:12:31.128630 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:31.128605 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:12:31.130227 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:31.130200 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:12:31.151657 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:31.151617 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:12:31.947520 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:31.947488 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:12:34.128289 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:34.128254 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:12:34.940764 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:12:34.940735 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:13:36.907623 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:36.907589 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b"] Sep 07 13:13:36.908178 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:36.907869 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="online" containerID="cri-o://e8fd977ed9a84b1081110c8177835e295bfae81a4692582147c8f6f5b576eef8" gracePeriod=30 Sep 07 13:13:36.908178 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:36.907971 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="ui" containerID="cri-o://830cf561186832fae7420e92aaf7ecb5d2b68eee47a871ce5341b27b993d4295" gracePeriod=30 Sep 07 13:13:36.908178 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:36.907985 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="offline" containerID="cri-o://437130537c93ae5da5a8924d0f0f0af41559cdadcdf3a79958b6dbc823dfcc05" gracePeriod=30 Sep 07 13:13:37.078136 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:37.078102 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr"] Sep 07 13:13:37.078598 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:37.078337 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="registry" containerID="cri-o://c2c42b20379956bd27bb708f9ea3fc9f92486e856bb981c3ca2bf986280e1a38" gracePeriod=30 Sep 07 13:13:37.078598 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:37.078419 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="ui" containerID="cri-o://557f6f4ba3114a8273812d41a3899476387926fc0bc21920d3d7b7b1173b2d87" gracePeriod=30 Sep 07 13:13:37.078598 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:37.078458 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="offline" containerID="cri-o://fa2b54e295a66c3f9a5ffca69abbf74f25b69c2fac518e2aed1ba06da981c012" gracePeriod=30 Sep 07 13:13:37.078598 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:37.078515 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="online" containerID="cri-o://2492435944a8dd37b8ab9fa788a842729653f5e51309c3e1f872b80d9ca822a4" gracePeriod=30 Sep 07 13:13:38.198903 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:38.198820 2564 generic.go:356] "Generic (PLEG): container finished" podID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerID="830cf561186832fae7420e92aaf7ecb5d2b68eee47a871ce5341b27b993d4295" exitCode=0 Sep 07 13:13:38.198903 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:38.198846 2564 generic.go:356] "Generic (PLEG): container finished" podID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerID="e8fd977ed9a84b1081110c8177835e295bfae81a4692582147c8f6f5b576eef8" exitCode=0 Sep 07 13:13:38.198903 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:38.198890 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" event={"ID":"af1211af-50e7-4a82-8da9-b703a5a3af99","Type":"ContainerDied","Data":"830cf561186832fae7420e92aaf7ecb5d2b68eee47a871ce5341b27b993d4295"} Sep 07 13:13:38.199511 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:38.198932 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" event={"ID":"af1211af-50e7-4a82-8da9-b703a5a3af99","Type":"ContainerDied","Data":"e8fd977ed9a84b1081110c8177835e295bfae81a4692582147c8f6f5b576eef8"} Sep 07 13:13:38.201733 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:38.201714 2564 generic.go:356] "Generic (PLEG): container finished" podID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerID="557f6f4ba3114a8273812d41a3899476387926fc0bc21920d3d7b7b1173b2d87" exitCode=0 Sep 07 13:13:38.201733 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:38.201731 2564 generic.go:356] "Generic (PLEG): container finished" podID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerID="2492435944a8dd37b8ab9fa788a842729653f5e51309c3e1f872b80d9ca822a4" exitCode=0 Sep 07 13:13:38.201862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:38.201775 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" event={"ID":"1712af5d-d5a3-4841-92cb-72109f23e22d","Type":"ContainerDied","Data":"557f6f4ba3114a8273812d41a3899476387926fc0bc21920d3d7b7b1173b2d87"} Sep 07 13:13:38.201862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:38.201801 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" event={"ID":"1712af5d-d5a3-4841-92cb-72109f23e22d","Type":"ContainerDied","Data":"2492435944a8dd37b8ab9fa788a842729653f5e51309c3e1f872b80d9ca822a4"} Sep 07 13:13:41.926592 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:41.926554 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 07 13:13:43.452544 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:43.452499 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Sep 07 13:13:44.940996 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:44.940951 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 07 13:13:46.464275 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:46.464232 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 07 13:13:51.926680 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:51.926633 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 07 13:13:53.452621 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:53.452572 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Sep 07 13:13:54.940700 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:54.940650 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 07 13:13:56.464451 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:13:56.464406 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 07 13:14:01.927228 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:01.927185 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 07 13:14:01.927623 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:01.927282 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:14:03.452294 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:03.452246 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Sep 07 13:14:03.452693 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:03.452368 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:14:04.941227 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:04.941186 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 07 13:14:04.941653 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:04.941308 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:14:06.464358 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:06.464311 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 07 13:14:06.464758 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:06.464465 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:14:07.321912 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.321882 2564 generic.go:356] "Generic (PLEG): container finished" podID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerID="fa2b54e295a66c3f9a5ffca69abbf74f25b69c2fac518e2aed1ba06da981c012" exitCode=137 Sep 07 13:14:07.321912 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.321908 2564 generic.go:356] "Generic (PLEG): container finished" podID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerID="c2c42b20379956bd27bb708f9ea3fc9f92486e856bb981c3ca2bf986280e1a38" exitCode=137 Sep 07 13:14:07.322128 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.321951 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" event={"ID":"1712af5d-d5a3-4841-92cb-72109f23e22d","Type":"ContainerDied","Data":"fa2b54e295a66c3f9a5ffca69abbf74f25b69c2fac518e2aed1ba06da981c012"} Sep 07 13:14:07.322128 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.321992 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" event={"ID":"1712af5d-d5a3-4841-92cb-72109f23e22d","Type":"ContainerDied","Data":"c2c42b20379956bd27bb708f9ea3fc9f92486e856bb981c3ca2bf986280e1a38"} Sep 07 13:14:07.324791 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.324762 2564 generic.go:356] "Generic (PLEG): container finished" podID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerID="437130537c93ae5da5a8924d0f0f0af41559cdadcdf3a79958b6dbc823dfcc05" exitCode=137 Sep 07 13:14:07.324906 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.324797 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" event={"ID":"af1211af-50e7-4a82-8da9-b703a5a3af99","Type":"ContainerDied","Data":"437130537c93ae5da5a8924d0f0f0af41559cdadcdf3a79958b6dbc823dfcc05"} Sep 07 13:14:07.563801 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.563775 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:14:07.653179 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.653152 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/af1211af-50e7-4a82-8da9-b703a5a3af99-registry-tls\") pod \"af1211af-50e7-4a82-8da9-b703a5a3af99\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " Sep 07 13:14:07.653306 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.653196 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/af1211af-50e7-4a82-8da9-b703a5a3af99-feast-data\") pod \"af1211af-50e7-4a82-8da9-b703a5a3af99\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " Sep 07 13:14:07.653306 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.653226 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96sgp\" (UniqueName: \"kubernetes.io/projected/af1211af-50e7-4a82-8da9-b703a5a3af99-kube-api-access-96sgp\") pod \"af1211af-50e7-4a82-8da9-b703a5a3af99\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " Sep 07 13:14:07.653417 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.653314 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-offline-tls\") pod \"af1211af-50e7-4a82-8da9-b703a5a3af99\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " Sep 07 13:14:07.653417 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.653355 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-online-tls\") pod \"af1211af-50e7-4a82-8da9-b703a5a3af99\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " Sep 07 13:14:07.653417 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.653389 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-ui-tls\") pod \"af1211af-50e7-4a82-8da9-b703a5a3af99\" (UID: \"af1211af-50e7-4a82-8da9-b703a5a3af99\") " Sep 07 13:14:07.653745 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.653700 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af1211af-50e7-4a82-8da9-b703a5a3af99-feast-data" (OuterVolumeSpecName: "feast-data") pod "af1211af-50e7-4a82-8da9-b703a5a3af99" (UID: "af1211af-50e7-4a82-8da9-b703a5a3af99"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:14:07.653858 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.653808 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af1211af-50e7-4a82-8da9-b703a5a3af99-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "af1211af-50e7-4a82-8da9-b703a5a3af99" (UID: "af1211af-50e7-4a82-8da9-b703a5a3af99"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 13:14:07.657229 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.657203 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-online-tls" (OuterVolumeSpecName: "online-tls") pod "af1211af-50e7-4a82-8da9-b703a5a3af99" (UID: "af1211af-50e7-4a82-8da9-b703a5a3af99"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:14:07.657348 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.657269 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "af1211af-50e7-4a82-8da9-b703a5a3af99" (UID: "af1211af-50e7-4a82-8da9-b703a5a3af99"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:14:07.657348 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.657311 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "af1211af-50e7-4a82-8da9-b703a5a3af99" (UID: "af1211af-50e7-4a82-8da9-b703a5a3af99"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:14:07.657348 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.657313 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af1211af-50e7-4a82-8da9-b703a5a3af99-kube-api-access-96sgp" (OuterVolumeSpecName: "kube-api-access-96sgp") pod "af1211af-50e7-4a82-8da9-b703a5a3af99" (UID: "af1211af-50e7-4a82-8da9-b703a5a3af99"). InnerVolumeSpecName "kube-api-access-96sgp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:14:07.731583 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.731560 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:14:07.754914 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.754885 2564 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-online-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:14:07.754914 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.754910 2564 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-ui-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:14:07.754914 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.754919 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/af1211af-50e7-4a82-8da9-b703a5a3af99-registry-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:14:07.755131 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.754927 2564 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/af1211af-50e7-4a82-8da9-b703a5a3af99-feast-data\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:14:07.755131 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.754935 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96sgp\" (UniqueName: \"kubernetes.io/projected/af1211af-50e7-4a82-8da9-b703a5a3af99-kube-api-access-96sgp\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:14:07.755131 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.754943 2564 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/af1211af-50e7-4a82-8da9-b703a5a3af99-offline-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:14:07.856042 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.855959 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-registry-tls\") pod \"1712af5d-d5a3-4841-92cb-72109f23e22d\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " Sep 07 13:14:07.856042 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.855994 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpcmw\" (UniqueName: \"kubernetes.io/projected/1712af5d-d5a3-4841-92cb-72109f23e22d-kube-api-access-qpcmw\") pod \"1712af5d-d5a3-4841-92cb-72109f23e22d\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " Sep 07 13:14:07.856042 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.856021 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-ui-tls\") pod \"1712af5d-d5a3-4841-92cb-72109f23e22d\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " Sep 07 13:14:07.856347 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.856074 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-offline-tls\") pod \"1712af5d-d5a3-4841-92cb-72109f23e22d\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " Sep 07 13:14:07.856347 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.856116 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-online-tls\") pod \"1712af5d-d5a3-4841-92cb-72109f23e22d\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " Sep 07 13:14:07.856347 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.856170 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1712af5d-d5a3-4841-92cb-72109f23e22d-feast-data\") pod \"1712af5d-d5a3-4841-92cb-72109f23e22d\" (UID: \"1712af5d-d5a3-4841-92cb-72109f23e22d\") " Sep 07 13:14:07.856718 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.856687 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1712af5d-d5a3-4841-92cb-72109f23e22d-feast-data" (OuterVolumeSpecName: "feast-data") pod "1712af5d-d5a3-4841-92cb-72109f23e22d" (UID: "1712af5d-d5a3-4841-92cb-72109f23e22d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:14:07.858121 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.858098 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1712af5d-d5a3-4841-92cb-72109f23e22d" (UID: "1712af5d-d5a3-4841-92cb-72109f23e22d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:14:07.858634 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.858606 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-online-tls" (OuterVolumeSpecName: "online-tls") pod "1712af5d-d5a3-4841-92cb-72109f23e22d" (UID: "1712af5d-d5a3-4841-92cb-72109f23e22d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:14:07.858634 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.858625 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "1712af5d-d5a3-4841-92cb-72109f23e22d" (UID: "1712af5d-d5a3-4841-92cb-72109f23e22d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:14:07.858756 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.858640 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "1712af5d-d5a3-4841-92cb-72109f23e22d" (UID: "1712af5d-d5a3-4841-92cb-72109f23e22d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:14:07.858756 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.858662 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1712af5d-d5a3-4841-92cb-72109f23e22d-kube-api-access-qpcmw" (OuterVolumeSpecName: "kube-api-access-qpcmw") pod "1712af5d-d5a3-4841-92cb-72109f23e22d" (UID: "1712af5d-d5a3-4841-92cb-72109f23e22d"). InnerVolumeSpecName "kube-api-access-qpcmw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:14:07.957634 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.957604 2564 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1712af5d-d5a3-4841-92cb-72109f23e22d-feast-data\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:14:07.957634 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.957632 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-registry-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:14:07.957795 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.957643 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpcmw\" (UniqueName: \"kubernetes.io/projected/1712af5d-d5a3-4841-92cb-72109f23e22d-kube-api-access-qpcmw\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:14:07.957795 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.957652 2564 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-ui-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:14:07.957795 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.957660 2564 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-offline-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:14:07.957795 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:07.957668 2564 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1712af5d-d5a3-4841-92cb-72109f23e22d-online-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:14:08.331731 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.331695 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" event={"ID":"1712af5d-d5a3-4841-92cb-72109f23e22d","Type":"ContainerDied","Data":"afd4cd5617cc941f4ca5632ea1a45786b787bea7cc3b0b4b68d4aa4d359db7a6"} Sep 07 13:14:08.331731 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.331719 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr" Sep 07 13:14:08.331995 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.331758 2564 scope.go:117] "RemoveContainer" containerID="557f6f4ba3114a8273812d41a3899476387926fc0bc21920d3d7b7b1173b2d87" Sep 07 13:14:08.335080 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.335029 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" event={"ID":"af1211af-50e7-4a82-8da9-b703a5a3af99","Type":"ContainerDied","Data":"8f79faca227e49c4648be5d10806ded825b5d1a932b380ee309ff634dfdc4d4c"} Sep 07 13:14:08.335080 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.335079 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b" Sep 07 13:14:08.344227 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.344192 2564 scope.go:117] "RemoveContainer" containerID="fa2b54e295a66c3f9a5ffca69abbf74f25b69c2fac518e2aed1ba06da981c012" Sep 07 13:14:08.352014 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.351998 2564 scope.go:117] "RemoveContainer" containerID="2492435944a8dd37b8ab9fa788a842729653f5e51309c3e1f872b80d9ca822a4" Sep 07 13:14:08.356929 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.356909 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr"] Sep 07 13:14:08.359819 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.359797 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-77b88b9d59-rhvkr"] Sep 07 13:14:08.360648 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.360634 2564 scope.go:117] "RemoveContainer" containerID="c2c42b20379956bd27bb708f9ea3fc9f92486e856bb981c3ca2bf986280e1a38" Sep 07 13:14:08.367912 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.367892 2564 scope.go:117] "RemoveContainer" containerID="a255c92cf984c20a19194a01c4ecca02180e10d338c8e215c66c8178d3ed9218" Sep 07 13:14:08.373722 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.373697 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b"] Sep 07 13:14:08.375785 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.375752 2564 scope.go:117] "RemoveContainer" containerID="55f471d340a45f1315b5e9776f6d7dc138bfef8ce6df4ea51a8cf7f3f8eb8e72" Sep 07 13:14:08.377577 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.377552 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-68df65b795-vsc5b"] Sep 07 13:14:08.382771 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.382756 2564 scope.go:117] "RemoveContainer" containerID="830cf561186832fae7420e92aaf7ecb5d2b68eee47a871ce5341b27b993d4295" Sep 07 13:14:08.389499 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.389483 2564 scope.go:117] "RemoveContainer" containerID="437130537c93ae5da5a8924d0f0f0af41559cdadcdf3a79958b6dbc823dfcc05" Sep 07 13:14:08.396758 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.396731 2564 scope.go:117] "RemoveContainer" containerID="e8fd977ed9a84b1081110c8177835e295bfae81a4692582147c8f6f5b576eef8" Sep 07 13:14:08.403833 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.403816 2564 scope.go:117] "RemoveContainer" containerID="6dd9fd99bc2b1ef32820ed8d8f4029bcf75abc9eb5b8a7ebd2fa3251a329bcee" Sep 07 13:14:08.410485 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:08.410470 2564 scope.go:117] "RemoveContainer" containerID="d898833f66cdb5c6834d63c7ee4415873ec93eac923130232c755a986c15a1f2" Sep 07 13:14:09.922738 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:09.922701 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" path="/var/lib/kubelet/pods/1712af5d-d5a3-4841-92cb-72109f23e22d/volumes" Sep 07 13:14:09.923327 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:09.923312 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" path="/var/lib/kubelet/pods/af1211af-50e7-4a82-8da9-b703a5a3af99/volumes" Sep 07 13:14:20.296012 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.295969 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-m4rx2"] Sep 07 13:14:20.296468 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296432 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="feast-apply" Sep 07 13:14:20.296468 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296452 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="feast-apply" Sep 07 13:14:20.296468 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296464 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="ui" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296475 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="ui" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296495 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="feast-init" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296505 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="feast-init" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296515 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="feast-apply" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296523 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="feast-apply" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296534 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="offline" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296542 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="offline" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296550 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="feast-init" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296559 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="feast-init" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296569 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="offline" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296587 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="offline" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296606 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="online" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296614 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="online" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296634 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="online" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296673 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="online" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296684 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="registry" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296693 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="registry" Sep 07 13:14:20.296704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296701 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="ui" Sep 07 13:14:20.297855 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296718 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="ui" Sep 07 13:14:20.297855 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296819 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="online" Sep 07 13:14:20.297855 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296833 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="offline" Sep 07 13:14:20.297855 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296840 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="offline" Sep 07 13:14:20.297855 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296849 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="online" Sep 07 13:14:20.297855 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296856 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="ui" Sep 07 13:14:20.297855 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296865 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1712af5d-d5a3-4841-92cb-72109f23e22d" containerName="registry" Sep 07 13:14:20.297855 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.296871 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="af1211af-50e7-4a82-8da9-b703a5a3af99" containerName="ui" Sep 07 13:14:20.301836 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.301813 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-m4rx2" Sep 07 13:14:20.304646 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.304626 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 07 13:14:20.304744 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.304648 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 07 13:14:20.304744 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.304666 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 07 13:14:20.305828 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.305625 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-zhgfh\"" Sep 07 13:14:20.307740 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.307717 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-m4rx2"] Sep 07 13:14:20.456044 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.455995 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/57b94e80-df27-4c8b-823e-83f747089b9a-postgresdata\") pod \"postgres-864846d4f5-m4rx2\" (UID: \"57b94e80-df27-4c8b-823e-83f747089b9a\") " pod="test-ns-feast/postgres-864846d4f5-m4rx2" Sep 07 13:14:20.456044 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.456070 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p5xz\" (UniqueName: \"kubernetes.io/projected/57b94e80-df27-4c8b-823e-83f747089b9a-kube-api-access-9p5xz\") pod \"postgres-864846d4f5-m4rx2\" (UID: \"57b94e80-df27-4c8b-823e-83f747089b9a\") " pod="test-ns-feast/postgres-864846d4f5-m4rx2" Sep 07 13:14:20.556832 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.556748 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/57b94e80-df27-4c8b-823e-83f747089b9a-postgresdata\") pod \"postgres-864846d4f5-m4rx2\" (UID: \"57b94e80-df27-4c8b-823e-83f747089b9a\") " pod="test-ns-feast/postgres-864846d4f5-m4rx2" Sep 07 13:14:20.556832 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.556789 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p5xz\" (UniqueName: \"kubernetes.io/projected/57b94e80-df27-4c8b-823e-83f747089b9a-kube-api-access-9p5xz\") pod \"postgres-864846d4f5-m4rx2\" (UID: \"57b94e80-df27-4c8b-823e-83f747089b9a\") " pod="test-ns-feast/postgres-864846d4f5-m4rx2" Sep 07 13:14:20.557188 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.557164 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/57b94e80-df27-4c8b-823e-83f747089b9a-postgresdata\") pod \"postgres-864846d4f5-m4rx2\" (UID: \"57b94e80-df27-4c8b-823e-83f747089b9a\") " pod="test-ns-feast/postgres-864846d4f5-m4rx2" Sep 07 13:14:20.566861 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.566837 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p5xz\" (UniqueName: \"kubernetes.io/projected/57b94e80-df27-4c8b-823e-83f747089b9a-kube-api-access-9p5xz\") pod \"postgres-864846d4f5-m4rx2\" (UID: \"57b94e80-df27-4c8b-823e-83f747089b9a\") " pod="test-ns-feast/postgres-864846d4f5-m4rx2" Sep 07 13:14:20.618030 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.617996 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-m4rx2" Sep 07 13:14:20.888670 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.888604 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-m4rx2"] Sep 07 13:14:20.891508 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:14:20.891481 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57b94e80_df27_4c8b_823e_83f747089b9a.slice/crio-4135bba1541a819b39d5bf2823fe480c345b7fd0fa27df1e9b989685b3a8ff57 WatchSource:0}: Error finding container 4135bba1541a819b39d5bf2823fe480c345b7fd0fa27df1e9b989685b3a8ff57: Status 404 returned error can't find the container with id 4135bba1541a819b39d5bf2823fe480c345b7fd0fa27df1e9b989685b3a8ff57 Sep 07 13:14:20.893513 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:20.893494 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 13:14:21.389933 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:21.389892 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-m4rx2" event={"ID":"57b94e80-df27-4c8b-823e-83f747089b9a","Type":"ContainerStarted","Data":"4135bba1541a819b39d5bf2823fe480c345b7fd0fa27df1e9b989685b3a8ff57"} Sep 07 13:14:26.413341 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:26.413302 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-m4rx2" event={"ID":"57b94e80-df27-4c8b-823e-83f747089b9a","Type":"ContainerStarted","Data":"65089c252a706605105c044512967e5d20976b6fa2bd98c99502337f6be729b2"} Sep 07 13:14:26.432761 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:26.432702 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-m4rx2" podStartSLOduration=1.669089587 podStartE2EDuration="6.432688073s" totalImagesPullingTime="4.763598486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:14:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:14:26.431905295 +0000 UTC m=+827.027409914" watchObservedRunningTime="2026-09-07 13:14:26.432688073 +0000 UTC m=+827.028192689" Sep 07 13:14:29.177788 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.177744 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww"] Sep 07 13:14:29.187032 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.186995 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.189687 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.189654 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Sep 07 13:14:29.189824 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.189722 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Sep 07 13:14:29.189824 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.189816 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Sep 07 13:14:29.189933 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.189831 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 07 13:14:29.189975 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.189931 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-pxq95\"" Sep 07 13:14:29.195817 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.195794 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww"] Sep 07 13:14:29.228229 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.228193 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-offline-tls\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.228229 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.228245 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-online-tls\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.228440 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.228311 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-feast-data\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.228440 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.228340 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89dbj\" (UniqueName: \"kubernetes.io/projected/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-kube-api-access-89dbj\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.228440 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.228373 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-registry-tls\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.329305 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.329263 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-feast-data\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.329483 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.329320 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89dbj\" (UniqueName: \"kubernetes.io/projected/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-kube-api-access-89dbj\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.329483 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.329347 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-registry-tls\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.329483 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.329382 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-offline-tls\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.329483 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.329411 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-online-tls\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.329621 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:14:29.329495 2564 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Sep 07 13:14:29.329621 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:14:29.329512 2564 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found Sep 07 13:14:29.329621 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:14:29.329579 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-online-tls podName:7d6ef589-aa97-40fd-a3bc-a5e59f0508ce nodeName:}" failed. No retries permitted until 2026-09-07 13:14:29.829559468 +0000 UTC m=+830.425064079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-online-tls") pod "feast-credit-scoring-cfb88dc86-x9zww" (UID: "7d6ef589-aa97-40fd-a3bc-a5e59f0508ce") : secret "feast-credit-scoring-online-tls" not found Sep 07 13:14:29.329621 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:14:29.329597 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-registry-tls podName:7d6ef589-aa97-40fd-a3bc-a5e59f0508ce nodeName:}" failed. No retries permitted until 2026-09-07 13:14:29.829588481 +0000 UTC m=+830.425093076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-registry-tls") pod "feast-credit-scoring-cfb88dc86-x9zww" (UID: "7d6ef589-aa97-40fd-a3bc-a5e59f0508ce") : secret "feast-credit-scoring-registry-tls" not found Sep 07 13:14:29.329790 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.329689 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-feast-data\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.331754 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.331735 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-offline-tls\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.340062 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.340029 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89dbj\" (UniqueName: \"kubernetes.io/projected/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-kube-api-access-89dbj\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.833846 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.833807 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-online-tls\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.834011 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.833876 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-registry-tls\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.836296 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.836264 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-registry-tls\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:29.836425 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:29.836344 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-online-tls\") pod \"feast-credit-scoring-cfb88dc86-x9zww\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:30.098142 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:30.098027 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:30.376177 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:30.376148 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww"] Sep 07 13:14:30.378264 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:14:30.378237 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d6ef589_aa97_40fd_a3bc_a5e59f0508ce.slice/crio-48655b9b9234974ef54b20eb530af875b284ace7f80ff4bcbb65992e815e0558 WatchSource:0}: Error finding container 48655b9b9234974ef54b20eb530af875b284ace7f80ff4bcbb65992e815e0558: Status 404 returned error can't find the container with id 48655b9b9234974ef54b20eb530af875b284ace7f80ff4bcbb65992e815e0558 Sep 07 13:14:30.430377 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:30.430345 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" event={"ID":"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce","Type":"ContainerStarted","Data":"48655b9b9234974ef54b20eb530af875b284ace7f80ff4bcbb65992e815e0558"} Sep 07 13:14:31.437564 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:31.437520 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" event={"ID":"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce","Type":"ContainerStarted","Data":"0e80e52dc8159c67ed4ac25e9d1c1b33048f71ae8502f08342860978d0de9e6e"} Sep 07 13:14:32.443928 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:32.443841 2564 generic.go:356] "Generic (PLEG): container finished" podID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerID="0e80e52dc8159c67ed4ac25e9d1c1b33048f71ae8502f08342860978d0de9e6e" exitCode=0 Sep 07 13:14:32.443928 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:32.443904 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" event={"ID":"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce","Type":"ContainerDied","Data":"0e80e52dc8159c67ed4ac25e9d1c1b33048f71ae8502f08342860978d0de9e6e"} Sep 07 13:14:33.449637 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:33.449599 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" event={"ID":"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce","Type":"ContainerStarted","Data":"8a68ae2c8b78d2d7447bafbe1e40aa576382a953d22452577fb65a29e82b410e"} Sep 07 13:14:39.477126 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:39.477093 2564 generic.go:356] "Generic (PLEG): container finished" podID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerID="8a68ae2c8b78d2d7447bafbe1e40aa576382a953d22452577fb65a29e82b410e" exitCode=0 Sep 07 13:14:39.477494 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:39.477164 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" event={"ID":"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce","Type":"ContainerDied","Data":"8a68ae2c8b78d2d7447bafbe1e40aa576382a953d22452577fb65a29e82b410e"} Sep 07 13:14:40.486186 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:40.486138 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" event={"ID":"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce","Type":"ContainerStarted","Data":"afca7cb6c2a3047678c21eff1a7ef1c81ef5898fc655d9cc2118f7c7bf5f7b77"} Sep 07 13:14:40.486675 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:40.486217 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" event={"ID":"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce","Type":"ContainerStarted","Data":"9d3c47e408c331907c0bafb38f1d621dc555be9073b414ec89cfc5ef2ab74bb6"} Sep 07 13:14:40.486675 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:40.486247 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" event={"ID":"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce","Type":"ContainerStarted","Data":"04345936ffa0b06b6bd4a795a4ae67bd4117cc4f2cefd4d41353f512d08510ae"} Sep 07 13:14:40.512988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:40.512924 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" podStartSLOduration=11.512902865000001 podStartE2EDuration="11.512902865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:14:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:14:40.510719635 +0000 UTC m=+841.106224252" watchObservedRunningTime="2026-09-07 13:14:40.512902865 +0000 UTC m=+841.108407483" Sep 07 13:14:42.098895 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:42.098775 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:42.098895 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:42.098867 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:42.098895 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:42.098894 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:42.100611 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:42.100569 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.30:8816: connect: connection refused" Sep 07 13:14:42.100760 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:42.100611 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 07 13:14:42.100760 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:42.100606 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.30:6571: connect: connection refused" Sep 07 13:14:45.099447 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:45.099380 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.30:8816: connect: connection refused" Sep 07 13:14:45.099447 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:45.099428 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 07 13:14:45.100009 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:45.099380 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.30:6571: connect: connection refused" Sep 07 13:14:48.098795 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:48.098740 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.30:6571: connect: connection refused" Sep 07 13:14:48.099786 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:48.099763 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:48.100191 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:48.100165 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:48.100673 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:48.100653 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:48.107126 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:48.107105 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:48.530552 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:48.530519 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:48.535233 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:48.535197 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:50.099126 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:50.099089 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:51.098631 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:51.098590 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.30:6571: connect: connection refused" Sep 07 13:14:54.099310 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:54.099277 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:14:54.100092 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:14:54.100074 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:15:59.411860 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.411819 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww"] Sep 07 13:15:59.412510 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.412124 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="registry" containerID="cri-o://04345936ffa0b06b6bd4a795a4ae67bd4117cc4f2cefd4d41353f512d08510ae" gracePeriod=30 Sep 07 13:15:59.412510 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.412220 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="online" containerID="cri-o://9d3c47e408c331907c0bafb38f1d621dc555be9073b414ec89cfc5ef2ab74bb6" gracePeriod=30 Sep 07 13:15:59.412510 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.412225 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="offline" containerID="cri-o://afca7cb6c2a3047678c21eff1a7ef1c81ef5898fc655d9cc2118f7c7bf5f7b77" gracePeriod=30 Sep 07 13:15:59.430626 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.430593 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-m4rx2"] Sep 07 13:15:59.430795 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.430762 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-m4rx2" podUID="57b94e80-df27-4c8b-823e-83f747089b9a" containerName="postgres" containerID="cri-o://65089c252a706605105c044512967e5d20976b6fa2bd98c99502337f6be729b2" gracePeriod=30 Sep 07 13:15:59.674539 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.674473 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-m4rx2" Sep 07 13:15:59.802255 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.802213 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9p5xz\" (UniqueName: \"kubernetes.io/projected/57b94e80-df27-4c8b-823e-83f747089b9a-kube-api-access-9p5xz\") pod \"57b94e80-df27-4c8b-823e-83f747089b9a\" (UID: \"57b94e80-df27-4c8b-823e-83f747089b9a\") " Sep 07 13:15:59.802459 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.802290 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/57b94e80-df27-4c8b-823e-83f747089b9a-postgresdata\") pod \"57b94e80-df27-4c8b-823e-83f747089b9a\" (UID: \"57b94e80-df27-4c8b-823e-83f747089b9a\") " Sep 07 13:15:59.804359 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.804322 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57b94e80-df27-4c8b-823e-83f747089b9a-kube-api-access-9p5xz" (OuterVolumeSpecName: "kube-api-access-9p5xz") pod "57b94e80-df27-4c8b-823e-83f747089b9a" (UID: "57b94e80-df27-4c8b-823e-83f747089b9a"). InnerVolumeSpecName "kube-api-access-9p5xz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:15:59.833720 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.833672 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57b94e80-df27-4c8b-823e-83f747089b9a-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "57b94e80-df27-4c8b-823e-83f747089b9a" (UID: "57b94e80-df27-4c8b-823e-83f747089b9a"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:15:59.834924 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.834890 2564 generic.go:356] "Generic (PLEG): container finished" podID="57b94e80-df27-4c8b-823e-83f747089b9a" containerID="65089c252a706605105c044512967e5d20976b6fa2bd98c99502337f6be729b2" exitCode=0 Sep 07 13:15:59.835037 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.834929 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-m4rx2" Sep 07 13:15:59.835037 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.834960 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-m4rx2" event={"ID":"57b94e80-df27-4c8b-823e-83f747089b9a","Type":"ContainerDied","Data":"65089c252a706605105c044512967e5d20976b6fa2bd98c99502337f6be729b2"} Sep 07 13:15:59.835037 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.835001 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-m4rx2" event={"ID":"57b94e80-df27-4c8b-823e-83f747089b9a","Type":"ContainerDied","Data":"4135bba1541a819b39d5bf2823fe480c345b7fd0fa27df1e9b989685b3a8ff57"} Sep 07 13:15:59.835037 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.835022 2564 scope.go:117] "RemoveContainer" containerID="65089c252a706605105c044512967e5d20976b6fa2bd98c99502337f6be729b2" Sep 07 13:15:59.844903 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.844877 2564 scope.go:117] "RemoveContainer" containerID="65089c252a706605105c044512967e5d20976b6fa2bd98c99502337f6be729b2" Sep 07 13:15:59.845299 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:15:59.845275 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65089c252a706605105c044512967e5d20976b6fa2bd98c99502337f6be729b2\": container with ID starting with 65089c252a706605105c044512967e5d20976b6fa2bd98c99502337f6be729b2 not found: ID does not exist" containerID="65089c252a706605105c044512967e5d20976b6fa2bd98c99502337f6be729b2" Sep 07 13:15:59.845377 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.845313 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65089c252a706605105c044512967e5d20976b6fa2bd98c99502337f6be729b2"} err="failed to get container status \"65089c252a706605105c044512967e5d20976b6fa2bd98c99502337f6be729b2\": rpc error: code = NotFound desc = could not find container \"65089c252a706605105c044512967e5d20976b6fa2bd98c99502337f6be729b2\": container with ID starting with 65089c252a706605105c044512967e5d20976b6fa2bd98c99502337f6be729b2 not found: ID does not exist" Sep 07 13:15:59.856494 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.856463 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-m4rx2"] Sep 07 13:15:59.860632 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.860606 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-m4rx2"] Sep 07 13:15:59.902974 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.902941 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9p5xz\" (UniqueName: \"kubernetes.io/projected/57b94e80-df27-4c8b-823e-83f747089b9a-kube-api-access-9p5xz\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:15:59.902974 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.902975 2564 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/57b94e80-df27-4c8b-823e-83f747089b9a-postgresdata\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:15:59.923124 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:15:59.923090 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57b94e80-df27-4c8b-823e-83f747089b9a" path="/var/lib/kubelet/pods/57b94e80-df27-4c8b-823e-83f747089b9a/volumes" Sep 07 13:16:00.843261 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:00.843229 2564 generic.go:356] "Generic (PLEG): container finished" podID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerID="9d3c47e408c331907c0bafb38f1d621dc555be9073b414ec89cfc5ef2ab74bb6" exitCode=0 Sep 07 13:16:00.843642 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:00.843279 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" event={"ID":"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce","Type":"ContainerDied","Data":"9d3c47e408c331907c0bafb38f1d621dc555be9073b414ec89cfc5ef2ab74bb6"} Sep 07 13:16:08.530980 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:08.530930 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 07 13:16:18.531581 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:18.531535 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 07 13:16:28.531325 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:28.531279 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Sep 07 13:16:28.531723 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:28.531398 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:16:29.962040 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:29.962005 2564 generic.go:356] "Generic (PLEG): container finished" podID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerID="afca7cb6c2a3047678c21eff1a7ef1c81ef5898fc655d9cc2118f7c7bf5f7b77" exitCode=137 Sep 07 13:16:29.962040 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:29.962032 2564 generic.go:356] "Generic (PLEG): container finished" podID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerID="04345936ffa0b06b6bd4a795a4ae67bd4117cc4f2cefd4d41353f512d08510ae" exitCode=137 Sep 07 13:16:29.962601 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:29.962081 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" event={"ID":"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce","Type":"ContainerDied","Data":"afca7cb6c2a3047678c21eff1a7ef1c81ef5898fc655d9cc2118f7c7bf5f7b77"} Sep 07 13:16:29.962601 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:29.962136 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" event={"ID":"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce","Type":"ContainerDied","Data":"04345936ffa0b06b6bd4a795a4ae67bd4117cc4f2cefd4d41353f512d08510ae"} Sep 07 13:16:30.064990 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.064968 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:16:30.125992 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.125960 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89dbj\" (UniqueName: \"kubernetes.io/projected/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-kube-api-access-89dbj\") pod \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " Sep 07 13:16:30.125992 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.125994 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-registry-tls\") pod \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " Sep 07 13:16:30.126340 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.126023 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-offline-tls\") pod \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " Sep 07 13:16:30.126340 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.126040 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-feast-data\") pod \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " Sep 07 13:16:30.126340 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.126169 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-online-tls\") pod \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\" (UID: \"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce\") " Sep 07 13:16:30.128222 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.128191 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" (UID: "7d6ef589-aa97-40fd-a3bc-a5e59f0508ce"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:16:30.128222 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.128212 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" (UID: "7d6ef589-aa97-40fd-a3bc-a5e59f0508ce"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:16:30.128387 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.128302 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-online-tls" (OuterVolumeSpecName: "online-tls") pod "7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" (UID: "7d6ef589-aa97-40fd-a3bc-a5e59f0508ce"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:16:30.128426 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.128414 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-kube-api-access-89dbj" (OuterVolumeSpecName: "kube-api-access-89dbj") pod "7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" (UID: "7d6ef589-aa97-40fd-a3bc-a5e59f0508ce"). InnerVolumeSpecName "kube-api-access-89dbj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:16:30.134624 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.134602 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-feast-data" (OuterVolumeSpecName: "feast-data") pod "7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" (UID: "7d6ef589-aa97-40fd-a3bc-a5e59f0508ce"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:16:30.227688 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.227661 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89dbj\" (UniqueName: \"kubernetes.io/projected/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-kube-api-access-89dbj\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:16:30.227688 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.227687 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-registry-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:16:30.227860 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.227697 2564 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-offline-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:16:30.227860 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.227704 2564 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-feast-data\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:16:30.227860 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.227712 2564 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce-online-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:16:30.968113 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.968084 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" Sep 07 13:16:30.968558 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.968082 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww" event={"ID":"7d6ef589-aa97-40fd-a3bc-a5e59f0508ce","Type":"ContainerDied","Data":"48655b9b9234974ef54b20eb530af875b284ace7f80ff4bcbb65992e815e0558"} Sep 07 13:16:30.968558 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.968228 2564 scope.go:117] "RemoveContainer" containerID="afca7cb6c2a3047678c21eff1a7ef1c81ef5898fc655d9cc2118f7c7bf5f7b77" Sep 07 13:16:30.977418 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.977401 2564 scope.go:117] "RemoveContainer" containerID="9d3c47e408c331907c0bafb38f1d621dc555be9073b414ec89cfc5ef2ab74bb6" Sep 07 13:16:30.984833 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.984812 2564 scope.go:117] "RemoveContainer" containerID="04345936ffa0b06b6bd4a795a4ae67bd4117cc4f2cefd4d41353f512d08510ae" Sep 07 13:16:30.991909 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.991892 2564 scope.go:117] "RemoveContainer" containerID="8a68ae2c8b78d2d7447bafbe1e40aa576382a953d22452577fb65a29e82b410e" Sep 07 13:16:30.993497 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.993477 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww"] Sep 07 13:16:30.998601 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:30.998579 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-cfb88dc86-x9zww"] Sep 07 13:16:31.000204 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:31.000183 2564 scope.go:117] "RemoveContainer" containerID="0e80e52dc8159c67ed4ac25e9d1c1b33048f71ae8502f08342860978d0de9e6e" Sep 07 13:16:31.921682 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:16:31.921646 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" path="/var/lib/kubelet/pods/7d6ef589-aa97-40fd-a3bc-a5e59f0508ce/volumes" Sep 07 13:18:35.552598 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552516 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-487ps"] Sep 07 13:18:35.553045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552844 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="registry" Sep 07 13:18:35.553045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552856 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="registry" Sep 07 13:18:35.553045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552867 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="feast-apply" Sep 07 13:18:35.553045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552873 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="feast-apply" Sep 07 13:18:35.553045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552879 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="offline" Sep 07 13:18:35.553045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552885 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="offline" Sep 07 13:18:35.553045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552898 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57b94e80-df27-4c8b-823e-83f747089b9a" containerName="postgres" Sep 07 13:18:35.553045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552903 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="57b94e80-df27-4c8b-823e-83f747089b9a" containerName="postgres" Sep 07 13:18:35.553045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552911 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="online" Sep 07 13:18:35.553045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552916 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="online" Sep 07 13:18:35.553045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552924 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="feast-init" Sep 07 13:18:35.553045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552929 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="feast-init" Sep 07 13:18:35.553045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552976 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="registry" Sep 07 13:18:35.553045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552983 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="57b94e80-df27-4c8b-823e-83f747089b9a" containerName="postgres" Sep 07 13:18:35.553045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552988 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="offline" Sep 07 13:18:35.553045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.552996 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d6ef589-aa97-40fd-a3bc-a5e59f0508ce" containerName="online" Sep 07 13:18:35.555802 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.555780 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-487ps" Sep 07 13:18:35.558512 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.558487 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 07 13:18:35.559372 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.559348 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-h8nwj\"" Sep 07 13:18:35.559479 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.559348 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 07 13:18:35.568785 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.568762 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-487ps"] Sep 07 13:18:35.653408 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.653372 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz8jg\" (UniqueName: \"kubernetes.io/projected/32db4bbd-bc9c-45b1-9a96-b9cf3e342732-kube-api-access-qz8jg\") pod \"redis-69bbd798b6-487ps\" (UID: \"32db4bbd-bc9c-45b1-9a96-b9cf3e342732\") " pod="test-ns-feast-rest/redis-69bbd798b6-487ps" Sep 07 13:18:35.753979 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.753932 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz8jg\" (UniqueName: \"kubernetes.io/projected/32db4bbd-bc9c-45b1-9a96-b9cf3e342732-kube-api-access-qz8jg\") pod \"redis-69bbd798b6-487ps\" (UID: \"32db4bbd-bc9c-45b1-9a96-b9cf3e342732\") " pod="test-ns-feast-rest/redis-69bbd798b6-487ps" Sep 07 13:18:35.763910 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.763877 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz8jg\" (UniqueName: \"kubernetes.io/projected/32db4bbd-bc9c-45b1-9a96-b9cf3e342732-kube-api-access-qz8jg\") pod \"redis-69bbd798b6-487ps\" (UID: \"32db4bbd-bc9c-45b1-9a96-b9cf3e342732\") " pod="test-ns-feast-rest/redis-69bbd798b6-487ps" Sep 07 13:18:35.875844 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:35.875739 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-487ps" Sep 07 13:18:36.161301 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:36.161276 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-487ps"] Sep 07 13:18:36.163770 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:18:36.163740 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32db4bbd_bc9c_45b1_9a96_b9cf3e342732.slice/crio-6d8e0aec4253f9d89636b39a45f4ececcd3ae84e471acd157c7ad73f9e0f6c92 WatchSource:0}: Error finding container 6d8e0aec4253f9d89636b39a45f4ececcd3ae84e471acd157c7ad73f9e0f6c92: Status 404 returned error can't find the container with id 6d8e0aec4253f9d89636b39a45f4ececcd3ae84e471acd157c7ad73f9e0f6c92 Sep 07 13:18:36.448100 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:36.447999 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-487ps" event={"ID":"32db4bbd-bc9c-45b1-9a96-b9cf3e342732","Type":"ContainerStarted","Data":"6d8e0aec4253f9d89636b39a45f4ececcd3ae84e471acd157c7ad73f9e0f6c92"} Sep 07 13:18:38.458658 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:38.458622 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-487ps" event={"ID":"32db4bbd-bc9c-45b1-9a96-b9cf3e342732","Type":"ContainerStarted","Data":"c530793caf05b230ae9eb580f24275f25f3807134c933b968d432358be5f7a09"} Sep 07 13:18:38.476333 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:38.476280 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-487ps" podStartSLOduration=1.93320125 podStartE2EDuration="3.476264704s" totalImagesPullingTime="1.543063454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:18:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:18:38.474902824 +0000 UTC m=+1079.070407441" watchObservedRunningTime="2026-09-07 13:18:38.476264704 +0000 UTC m=+1079.071769320" Sep 07 13:18:41.029188 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:41.029152 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-57ttw"] Sep 07 13:18:41.032387 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:41.032358 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-57ttw" Sep 07 13:18:41.035063 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:41.035033 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 07 13:18:41.039987 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:41.039937 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-57ttw"] Sep 07 13:18:41.094135 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:41.094094 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1621decd-bc38-4243-8f70-0873c25a339d-postgresdata\") pod \"postgres-864846d4f5-57ttw\" (UID: \"1621decd-bc38-4243-8f70-0873c25a339d\") " pod="test-ns-feast-rest/postgres-864846d4f5-57ttw" Sep 07 13:18:41.094315 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:41.094166 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs2cg\" (UniqueName: \"kubernetes.io/projected/1621decd-bc38-4243-8f70-0873c25a339d-kube-api-access-fs2cg\") pod \"postgres-864846d4f5-57ttw\" (UID: \"1621decd-bc38-4243-8f70-0873c25a339d\") " pod="test-ns-feast-rest/postgres-864846d4f5-57ttw" Sep 07 13:18:41.195082 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:41.195008 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1621decd-bc38-4243-8f70-0873c25a339d-postgresdata\") pod \"postgres-864846d4f5-57ttw\" (UID: \"1621decd-bc38-4243-8f70-0873c25a339d\") " pod="test-ns-feast-rest/postgres-864846d4f5-57ttw" Sep 07 13:18:41.195273 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:41.195130 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs2cg\" (UniqueName: \"kubernetes.io/projected/1621decd-bc38-4243-8f70-0873c25a339d-kube-api-access-fs2cg\") pod \"postgres-864846d4f5-57ttw\" (UID: \"1621decd-bc38-4243-8f70-0873c25a339d\") " pod="test-ns-feast-rest/postgres-864846d4f5-57ttw" Sep 07 13:18:41.195407 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:41.195384 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1621decd-bc38-4243-8f70-0873c25a339d-postgresdata\") pod \"postgres-864846d4f5-57ttw\" (UID: \"1621decd-bc38-4243-8f70-0873c25a339d\") " pod="test-ns-feast-rest/postgres-864846d4f5-57ttw" Sep 07 13:18:41.205581 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:41.205550 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs2cg\" (UniqueName: \"kubernetes.io/projected/1621decd-bc38-4243-8f70-0873c25a339d-kube-api-access-fs2cg\") pod \"postgres-864846d4f5-57ttw\" (UID: \"1621decd-bc38-4243-8f70-0873c25a339d\") " pod="test-ns-feast-rest/postgres-864846d4f5-57ttw" Sep 07 13:18:41.342931 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:41.342823 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-57ttw" Sep 07 13:18:41.617179 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:41.617152 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-57ttw"] Sep 07 13:18:41.619532 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:18:41.619507 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1621decd_bc38_4243_8f70_0873c25a339d.slice/crio-a9046ba5a77e275b58bda96d61b20ab8ea1d5b76ba1ac016651a62bd89cd6774 WatchSource:0}: Error finding container a9046ba5a77e275b58bda96d61b20ab8ea1d5b76ba1ac016651a62bd89cd6774: Status 404 returned error can't find the container with id a9046ba5a77e275b58bda96d61b20ab8ea1d5b76ba1ac016651a62bd89cd6774 Sep 07 13:18:42.477416 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:42.477372 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-57ttw" event={"ID":"1621decd-bc38-4243-8f70-0873c25a339d","Type":"ContainerStarted","Data":"e2fdaa995bc99b719a83aa61aa2c1a4fbb078bf26d5edcb39ac4d922aa550f54"} Sep 07 13:18:42.477416 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:42.477422 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-57ttw" event={"ID":"1621decd-bc38-4243-8f70-0873c25a339d","Type":"ContainerStarted","Data":"a9046ba5a77e275b58bda96d61b20ab8ea1d5b76ba1ac016651a62bd89cd6774"} Sep 07 13:18:46.760318 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.760250 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-57ttw" podStartSLOduration=5.7602333439999995 podStartE2EDuration="5.760233344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:18:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:18:42.494958839 +0000 UTC m=+1083.090463458" watchObservedRunningTime="2026-09-07 13:18:46.760233344 +0000 UTC m=+1087.355737962" Sep 07 13:18:46.761731 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.761707 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm"] Sep 07 13:18:46.768200 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.768177 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:18:46.771885 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.771862 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Sep 07 13:18:46.772200 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.772173 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-9stgt\"" Sep 07 13:18:46.772200 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.772193 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Sep 07 13:18:46.772364 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.772328 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 07 13:18:46.787124 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.787091 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm"] Sep 07 13:18:46.842504 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.842463 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rszbs\" (UniqueName: \"kubernetes.io/projected/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-kube-api-access-rszbs\") pod \"feast-credit-scoring-f9c7bbf4f-dsdjm\" (UID: \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\") " pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:18:46.842693 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.842542 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-online-tls\") pod \"feast-credit-scoring-f9c7bbf4f-dsdjm\" (UID: \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\") " pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:18:46.842693 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.842562 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-feast-data\") pod \"feast-credit-scoring-f9c7bbf4f-dsdjm\" (UID: \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\") " pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:18:46.842693 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.842623 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-registry-tls\") pod \"feast-credit-scoring-f9c7bbf4f-dsdjm\" (UID: \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\") " pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:18:46.943644 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.943608 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-online-tls\") pod \"feast-credit-scoring-f9c7bbf4f-dsdjm\" (UID: \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\") " pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:18:46.943644 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.943646 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-feast-data\") pod \"feast-credit-scoring-f9c7bbf4f-dsdjm\" (UID: \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\") " pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:18:46.943913 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.943667 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-registry-tls\") pod \"feast-credit-scoring-f9c7bbf4f-dsdjm\" (UID: \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\") " pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:18:46.943913 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.943721 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rszbs\" (UniqueName: \"kubernetes.io/projected/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-kube-api-access-rszbs\") pod \"feast-credit-scoring-f9c7bbf4f-dsdjm\" (UID: \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\") " pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:18:46.944079 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.944032 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-feast-data\") pod \"feast-credit-scoring-f9c7bbf4f-dsdjm\" (UID: \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\") " pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:18:46.946151 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.946122 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-online-tls\") pod \"feast-credit-scoring-f9c7bbf4f-dsdjm\" (UID: \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\") " pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:18:46.946251 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.946169 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-registry-tls\") pod \"feast-credit-scoring-f9c7bbf4f-dsdjm\" (UID: \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\") " pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:18:46.955699 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:46.955672 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rszbs\" (UniqueName: \"kubernetes.io/projected/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-kube-api-access-rszbs\") pod \"feast-credit-scoring-f9c7bbf4f-dsdjm\" (UID: \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\") " pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:18:47.078280 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:47.078192 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:18:47.391497 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:47.391471 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm"] Sep 07 13:18:47.393926 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:18:47.393897 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod734b5f5a_5ce3_474c_89ad_81eadf32bc2e.slice/crio-fc06119569a61459a982af4303f48305a82028aa5f1c98f45c074110db19d651 WatchSource:0}: Error finding container fc06119569a61459a982af4303f48305a82028aa5f1c98f45c074110db19d651: Status 404 returned error can't find the container with id fc06119569a61459a982af4303f48305a82028aa5f1c98f45c074110db19d651 Sep 07 13:18:47.498640 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:47.498604 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" event={"ID":"734b5f5a-5ce3-474c-89ad-81eadf32bc2e","Type":"ContainerStarted","Data":"2fbea6d52d9d2d9016b689d124c20e285ba1294d0fea495efa56d74b34173609"} Sep 07 13:18:47.498807 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:47.498662 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" event={"ID":"734b5f5a-5ce3-474c-89ad-81eadf32bc2e","Type":"ContainerStarted","Data":"fc06119569a61459a982af4303f48305a82028aa5f1c98f45c074110db19d651"} Sep 07 13:18:49.508777 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:49.508747 2564 generic.go:356] "Generic (PLEG): container finished" podID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerID="2fbea6d52d9d2d9016b689d124c20e285ba1294d0fea495efa56d74b34173609" exitCode=0 Sep 07 13:18:49.509155 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:49.508823 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" event={"ID":"734b5f5a-5ce3-474c-89ad-81eadf32bc2e","Type":"ContainerDied","Data":"2fbea6d52d9d2d9016b689d124c20e285ba1294d0fea495efa56d74b34173609"} Sep 07 13:18:50.515125 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:50.515086 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" event={"ID":"734b5f5a-5ce3-474c-89ad-81eadf32bc2e","Type":"ContainerStarted","Data":"c02ccc68565cc6b7a6ccda92c36bc0a1b633a8381804c3e09a87da8d1c277ec0"} Sep 07 13:18:57.546099 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:57.546047 2564 generic.go:356] "Generic (PLEG): container finished" podID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerID="c02ccc68565cc6b7a6ccda92c36bc0a1b633a8381804c3e09a87da8d1c277ec0" exitCode=0 Sep 07 13:18:57.546580 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:57.546119 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" event={"ID":"734b5f5a-5ce3-474c-89ad-81eadf32bc2e","Type":"ContainerDied","Data":"c02ccc68565cc6b7a6ccda92c36bc0a1b633a8381804c3e09a87da8d1c277ec0"} Sep 07 13:18:58.553551 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:58.553509 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" event={"ID":"734b5f5a-5ce3-474c-89ad-81eadf32bc2e","Type":"ContainerStarted","Data":"6981bfe74f12cdb0847074aa58bf95fa188dd6ff7855add818854f1dbb263dbc"} Sep 07 13:18:58.553989 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:58.553577 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" event={"ID":"734b5f5a-5ce3-474c-89ad-81eadf32bc2e","Type":"ContainerStarted","Data":"292d1811b5a1a3b45e7fd6a912a095b7012df61967887afaa41c4cec09beefc2"} Sep 07 13:18:58.583461 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:58.583400 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" podStartSLOduration=12.583382752 podStartE2EDuration="12.583382752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:18:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:18:58.581231658 +0000 UTC m=+1099.176736310" watchObservedRunningTime="2026-09-07 13:18:58.583382752 +0000 UTC m=+1099.178887368" Sep 07 13:18:59.079376 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:59.079329 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:18:59.079536 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:59.079410 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:18:59.081066 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:59.081014 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.33:6571: connect: connection refused" Sep 07 13:18:59.081202 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:18:59.081088 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Sep 07 13:19:02.079496 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:02.079449 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.33:6571: connect: connection refused" Sep 07 13:19:02.079918 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:02.079451 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Sep 07 13:19:05.078924 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:05.078869 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.33:6571: connect: connection refused" Sep 07 13:19:05.085010 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:05.084976 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:19:05.085599 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:05.085579 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:19:05.088670 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:05.088644 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:19:07.078712 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:07.078681 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:19:08.079341 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:08.079298 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.33:6571: connect: connection refused" Sep 07 13:19:11.080015 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:11.079978 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:19:11.080812 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:11.080790 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:19:12.585240 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.585197 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx"] Sep 07 13:19:12.588704 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.588683 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:12.591471 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.591436 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Sep 07 13:19:12.591607 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.591508 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-m66dj\"" Sep 07 13:19:12.592081 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.592046 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Sep 07 13:19:12.592140 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.592071 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Sep 07 13:19:12.600923 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.600894 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx"] Sep 07 13:19:12.777817 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.777780 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9728dd62-61bd-4385-9c96-5b8a63b2cef7-feast-data\") pod \"feast-driver-ranking-78b9b4767d-fnnvx\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:12.778083 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.777838 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9728dd62-61bd-4385-9c96-5b8a63b2cef7-online-tls\") pod \"feast-driver-ranking-78b9b4767d-fnnvx\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:12.778083 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.777868 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6qf8\" (UniqueName: \"kubernetes.io/projected/9728dd62-61bd-4385-9c96-5b8a63b2cef7-kube-api-access-j6qf8\") pod \"feast-driver-ranking-78b9b4767d-fnnvx\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:12.778083 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.777909 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9728dd62-61bd-4385-9c96-5b8a63b2cef7-registry-tls\") pod \"feast-driver-ranking-78b9b4767d-fnnvx\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:12.879449 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.879350 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9728dd62-61bd-4385-9c96-5b8a63b2cef7-registry-tls\") pod \"feast-driver-ranking-78b9b4767d-fnnvx\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:12.879449 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.879429 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9728dd62-61bd-4385-9c96-5b8a63b2cef7-feast-data\") pod \"feast-driver-ranking-78b9b4767d-fnnvx\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:12.879658 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.879481 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9728dd62-61bd-4385-9c96-5b8a63b2cef7-online-tls\") pod \"feast-driver-ranking-78b9b4767d-fnnvx\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:12.879658 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.879503 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6qf8\" (UniqueName: \"kubernetes.io/projected/9728dd62-61bd-4385-9c96-5b8a63b2cef7-kube-api-access-j6qf8\") pod \"feast-driver-ranking-78b9b4767d-fnnvx\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:12.879658 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:19:12.879516 2564 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-registry-tls: secret "feast-driver-ranking-registry-tls" not found Sep 07 13:19:12.879658 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:19:12.879601 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9728dd62-61bd-4385-9c96-5b8a63b2cef7-registry-tls podName:9728dd62-61bd-4385-9c96-5b8a63b2cef7 nodeName:}" failed. No retries permitted until 2026-09-07 13:19:13.379577841 +0000 UTC m=+1113.975082445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/9728dd62-61bd-4385-9c96-5b8a63b2cef7-registry-tls") pod "feast-driver-ranking-78b9b4767d-fnnvx" (UID: "9728dd62-61bd-4385-9c96-5b8a63b2cef7") : secret "feast-driver-ranking-registry-tls" not found Sep 07 13:19:12.879928 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.879894 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9728dd62-61bd-4385-9c96-5b8a63b2cef7-feast-data\") pod \"feast-driver-ranking-78b9b4767d-fnnvx\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:12.881960 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.881932 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9728dd62-61bd-4385-9c96-5b8a63b2cef7-online-tls\") pod \"feast-driver-ranking-78b9b4767d-fnnvx\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:12.889142 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:12.889111 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6qf8\" (UniqueName: \"kubernetes.io/projected/9728dd62-61bd-4385-9c96-5b8a63b2cef7-kube-api-access-j6qf8\") pod \"feast-driver-ranking-78b9b4767d-fnnvx\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:13.384131 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:13.384084 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9728dd62-61bd-4385-9c96-5b8a63b2cef7-registry-tls\") pod \"feast-driver-ranking-78b9b4767d-fnnvx\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:13.386740 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:13.386704 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9728dd62-61bd-4385-9c96-5b8a63b2cef7-registry-tls\") pod \"feast-driver-ranking-78b9b4767d-fnnvx\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:13.497814 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:13.497773 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:13.774493 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:13.774421 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx"] Sep 07 13:19:13.776647 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:19:13.776618 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9728dd62_61bd_4385_9c96_5b8a63b2cef7.slice/crio-3fba784e2e6f902f6379f83f04f9684996c571b7a979efb3fcb020e396084716 WatchSource:0}: Error finding container 3fba784e2e6f902f6379f83f04f9684996c571b7a979efb3fcb020e396084716: Status 404 returned error can't find the container with id 3fba784e2e6f902f6379f83f04f9684996c571b7a979efb3fcb020e396084716 Sep 07 13:19:14.629384 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:14.629343 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" event={"ID":"9728dd62-61bd-4385-9c96-5b8a63b2cef7","Type":"ContainerStarted","Data":"3b19fd94ebd6e18473e279ec4ca012c4d4c94da98d3758f69203ab89de8dd549"} Sep 07 13:19:14.629580 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:14.629404 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" event={"ID":"9728dd62-61bd-4385-9c96-5b8a63b2cef7","Type":"ContainerStarted","Data":"3fba784e2e6f902f6379f83f04f9684996c571b7a979efb3fcb020e396084716"} Sep 07 13:19:19.655226 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:19.655190 2564 generic.go:356] "Generic (PLEG): container finished" podID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerID="3b19fd94ebd6e18473e279ec4ca012c4d4c94da98d3758f69203ab89de8dd549" exitCode=0 Sep 07 13:19:19.655606 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:19.655252 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" event={"ID":"9728dd62-61bd-4385-9c96-5b8a63b2cef7","Type":"ContainerDied","Data":"3b19fd94ebd6e18473e279ec4ca012c4d4c94da98d3758f69203ab89de8dd549"} Sep 07 13:19:20.661106 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:20.661074 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" event={"ID":"9728dd62-61bd-4385-9c96-5b8a63b2cef7","Type":"ContainerStarted","Data":"f28130cf55d9eaac8ec7db257b2445657f429461bf8d633177fe065b2e0615ce"} Sep 07 13:19:27.693460 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:27.693419 2564 generic.go:356] "Generic (PLEG): container finished" podID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerID="f28130cf55d9eaac8ec7db257b2445657f429461bf8d633177fe065b2e0615ce" exitCode=0 Sep 07 13:19:27.693858 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:27.693491 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" event={"ID":"9728dd62-61bd-4385-9c96-5b8a63b2cef7","Type":"ContainerDied","Data":"f28130cf55d9eaac8ec7db257b2445657f429461bf8d633177fe065b2e0615ce"} Sep 07 13:19:28.701031 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:28.700984 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" event={"ID":"9728dd62-61bd-4385-9c96-5b8a63b2cef7","Type":"ContainerStarted","Data":"d667ef1d157723c1866086c262e0838f6dfe8f20ade68c87846f025138486dc0"} Sep 07 13:19:28.701439 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:28.701085 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" event={"ID":"9728dd62-61bd-4385-9c96-5b8a63b2cef7","Type":"ContainerStarted","Data":"97b62e4dc281435ffd8984744b93bedcb640609d6338478eb9313106c34599e7"} Sep 07 13:19:28.727465 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:28.727396 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" podStartSLOduration=16.72737282 podStartE2EDuration="16.72737282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 13:19:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:19:28.726756413 +0000 UTC m=+1129.322261045" watchObservedRunningTime="2026-09-07 13:19:28.72737282 +0000 UTC m=+1129.322877437" Sep 07 13:19:31.498928 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:31.498882 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:31.499437 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:31.499005 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:31.500733 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:31.500698 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.34:6571: connect: connection refused" Sep 07 13:19:31.500862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:31.500713 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Sep 07 13:19:33.498254 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:33.498209 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:33.498793 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:33.498307 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:34.499167 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:34.499112 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.34:6571: connect: connection refused" Sep 07 13:19:34.505281 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:34.505248 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:34.508908 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:34.508881 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:37.498194 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:37.498144 2564 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.34:6571: connect: connection refused" Sep 07 13:19:40.499285 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:40.499250 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:19:40.499975 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:19:40.499953 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:21:02.358159 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:02.358119 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx"] Sep 07 13:21:02.358688 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:02.358322 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx" podUID="50f7a4f8-db11-4632-bd06-4e689939d4df" containerName="manager" containerID="cri-o://c0f8ffe6ab8dafa16e35f4d0d5efabb0ff9f67bcef879661ca96cae586ca4d7c" gracePeriod=10 Sep 07 13:21:02.626265 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:02.626238 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx" Sep 07 13:21:02.633863 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:02.633839 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxzm6\" (UniqueName: \"kubernetes.io/projected/50f7a4f8-db11-4632-bd06-4e689939d4df-kube-api-access-bxzm6\") pod \"50f7a4f8-db11-4632-bd06-4e689939d4df\" (UID: \"50f7a4f8-db11-4632-bd06-4e689939d4df\") " Sep 07 13:21:02.636308 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:02.636274 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f7a4f8-db11-4632-bd06-4e689939d4df-kube-api-access-bxzm6" (OuterVolumeSpecName: "kube-api-access-bxzm6") pod "50f7a4f8-db11-4632-bd06-4e689939d4df" (UID: "50f7a4f8-db11-4632-bd06-4e689939d4df"). InnerVolumeSpecName "kube-api-access-bxzm6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:21:02.734605 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:02.734571 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxzm6\" (UniqueName: \"kubernetes.io/projected/50f7a4f8-db11-4632-bd06-4e689939d4df-kube-api-access-bxzm6\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:21:02.788619 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:02.788583 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm"] Sep 07 13:21:02.788844 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:02.788813 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="registry" containerID="cri-o://292d1811b5a1a3b45e7fd6a912a095b7012df61967887afaa41c4cec09beefc2" gracePeriod=30 Sep 07 13:21:02.788963 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:02.788855 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="online" containerID="cri-o://6981bfe74f12cdb0847074aa58bf95fa188dd6ff7855add818854f1dbb263dbc" gracePeriod=30 Sep 07 13:21:02.815360 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:02.815320 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx"] Sep 07 13:21:02.815605 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:02.815582 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="registry" containerID="cri-o://97b62e4dc281435ffd8984744b93bedcb640609d6338478eb9313106c34599e7" gracePeriod=30 Sep 07 13:21:02.815694 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:02.815624 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="online" containerID="cri-o://d667ef1d157723c1866086c262e0838f6dfe8f20ade68c87846f025138486dc0" gracePeriod=30 Sep 07 13:21:03.126677 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:03.126635 2564 generic.go:356] "Generic (PLEG): container finished" podID="50f7a4f8-db11-4632-bd06-4e689939d4df" containerID="c0f8ffe6ab8dafa16e35f4d0d5efabb0ff9f67bcef879661ca96cae586ca4d7c" exitCode=0 Sep 07 13:21:03.126862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:03.126693 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx" Sep 07 13:21:03.126862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:03.126728 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx" event={"ID":"50f7a4f8-db11-4632-bd06-4e689939d4df","Type":"ContainerDied","Data":"c0f8ffe6ab8dafa16e35f4d0d5efabb0ff9f67bcef879661ca96cae586ca4d7c"} Sep 07 13:21:03.126862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:03.126796 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx" event={"ID":"50f7a4f8-db11-4632-bd06-4e689939d4df","Type":"ContainerDied","Data":"5de7dfa526954cc78f9b47ead53cb1c32e0cde4d843b277a2266282f0aa0c995"} Sep 07 13:21:03.126862 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:03.126830 2564 scope.go:117] "RemoveContainer" containerID="c0f8ffe6ab8dafa16e35f4d0d5efabb0ff9f67bcef879661ca96cae586ca4d7c" Sep 07 13:21:03.136746 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:03.136721 2564 scope.go:117] "RemoveContainer" containerID="c0f8ffe6ab8dafa16e35f4d0d5efabb0ff9f67bcef879661ca96cae586ca4d7c" Sep 07 13:21:03.137143 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:21:03.137117 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0f8ffe6ab8dafa16e35f4d0d5efabb0ff9f67bcef879661ca96cae586ca4d7c\": container with ID starting with c0f8ffe6ab8dafa16e35f4d0d5efabb0ff9f67bcef879661ca96cae586ca4d7c not found: ID does not exist" containerID="c0f8ffe6ab8dafa16e35f4d0d5efabb0ff9f67bcef879661ca96cae586ca4d7c" Sep 07 13:21:03.137212 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:03.137154 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0f8ffe6ab8dafa16e35f4d0d5efabb0ff9f67bcef879661ca96cae586ca4d7c"} err="failed to get container status \"c0f8ffe6ab8dafa16e35f4d0d5efabb0ff9f67bcef879661ca96cae586ca4d7c\": rpc error: code = NotFound desc = could not find container \"c0f8ffe6ab8dafa16e35f4d0d5efabb0ff9f67bcef879661ca96cae586ca4d7c\": container with ID starting with c0f8ffe6ab8dafa16e35f4d0d5efabb0ff9f67bcef879661ca96cae586ca4d7c not found: ID does not exist" Sep 07 13:21:03.150443 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:03.150414 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx"] Sep 07 13:21:03.154654 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:03.154626 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-784cdbd879-k5hqx"] Sep 07 13:21:03.922937 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:03.922854 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50f7a4f8-db11-4632-bd06-4e689939d4df" path="/var/lib/kubelet/pods/50f7a4f8-db11-4632-bd06-4e689939d4df/volumes" Sep 07 13:21:04.133719 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:04.133679 2564 generic.go:356] "Generic (PLEG): container finished" podID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerID="6981bfe74f12cdb0847074aa58bf95fa188dd6ff7855add818854f1dbb263dbc" exitCode=0 Sep 07 13:21:04.133938 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:04.133754 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" event={"ID":"734b5f5a-5ce3-474c-89ad-81eadf32bc2e","Type":"ContainerDied","Data":"6981bfe74f12cdb0847074aa58bf95fa188dd6ff7855add818854f1dbb263dbc"} Sep 07 13:21:04.136752 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:04.136730 2564 generic.go:356] "Generic (PLEG): container finished" podID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerID="d667ef1d157723c1866086c262e0838f6dfe8f20ade68c87846f025138486dc0" exitCode=0 Sep 07 13:21:04.136877 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:04.136760 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" event={"ID":"9728dd62-61bd-4385-9c96-5b8a63b2cef7","Type":"ContainerDied","Data":"d667ef1d157723c1866086c262e0838f6dfe8f20ade68c87846f025138486dc0"} Sep 07 13:21:04.506303 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:04.506261 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Sep 07 13:21:04.706247 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:04.706209 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-57ttw"] Sep 07 13:21:04.706437 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:04.706397 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-57ttw" podUID="1621decd-bc38-4243-8f70-0873c25a339d" containerName="postgres" containerID="cri-o://e2fdaa995bc99b719a83aa61aa2c1a4fbb078bf26d5edcb39ac4d922aa550f54" gracePeriod=30 Sep 07 13:21:04.710790 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:04.710764 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-487ps"] Sep 07 13:21:04.710946 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:04.710926 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-487ps" podUID="32db4bbd-bc9c-45b1-9a96-b9cf3e342732" containerName="redis" containerID="cri-o://c530793caf05b230ae9eb580f24275f25f3807134c933b968d432358be5f7a09" gracePeriod=30 Sep 07 13:21:04.972482 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:04.972449 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-57ttw" Sep 07 13:21:04.975101 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:04.975082 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-487ps" Sep 07 13:21:05.049441 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.049349 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qz8jg\" (UniqueName: \"kubernetes.io/projected/32db4bbd-bc9c-45b1-9a96-b9cf3e342732-kube-api-access-qz8jg\") pod \"32db4bbd-bc9c-45b1-9a96-b9cf3e342732\" (UID: \"32db4bbd-bc9c-45b1-9a96-b9cf3e342732\") " Sep 07 13:21:05.049441 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.049412 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1621decd-bc38-4243-8f70-0873c25a339d-postgresdata\") pod \"1621decd-bc38-4243-8f70-0873c25a339d\" (UID: \"1621decd-bc38-4243-8f70-0873c25a339d\") " Sep 07 13:21:05.049668 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.049460 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fs2cg\" (UniqueName: \"kubernetes.io/projected/1621decd-bc38-4243-8f70-0873c25a339d-kube-api-access-fs2cg\") pod \"1621decd-bc38-4243-8f70-0873c25a339d\" (UID: \"1621decd-bc38-4243-8f70-0873c25a339d\") " Sep 07 13:21:05.051755 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.051714 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1621decd-bc38-4243-8f70-0873c25a339d-kube-api-access-fs2cg" (OuterVolumeSpecName: "kube-api-access-fs2cg") pod "1621decd-bc38-4243-8f70-0873c25a339d" (UID: "1621decd-bc38-4243-8f70-0873c25a339d"). InnerVolumeSpecName "kube-api-access-fs2cg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:21:05.051755 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.051729 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32db4bbd-bc9c-45b1-9a96-b9cf3e342732-kube-api-access-qz8jg" (OuterVolumeSpecName: "kube-api-access-qz8jg") pod "32db4bbd-bc9c-45b1-9a96-b9cf3e342732" (UID: "32db4bbd-bc9c-45b1-9a96-b9cf3e342732"). InnerVolumeSpecName "kube-api-access-qz8jg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:21:05.079827 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.079786 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1621decd-bc38-4243-8f70-0873c25a339d-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "1621decd-bc38-4243-8f70-0873c25a339d" (UID: "1621decd-bc38-4243-8f70-0873c25a339d"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:21:05.086194 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.086166 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Sep 07 13:21:05.141976 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.141940 2564 generic.go:356] "Generic (PLEG): container finished" podID="32db4bbd-bc9c-45b1-9a96-b9cf3e342732" containerID="c530793caf05b230ae9eb580f24275f25f3807134c933b968d432358be5f7a09" exitCode=0 Sep 07 13:21:05.142164 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.141977 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-487ps" Sep 07 13:21:05.142164 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.142031 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-487ps" event={"ID":"32db4bbd-bc9c-45b1-9a96-b9cf3e342732","Type":"ContainerDied","Data":"c530793caf05b230ae9eb580f24275f25f3807134c933b968d432358be5f7a09"} Sep 07 13:21:05.142164 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.142091 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-487ps" event={"ID":"32db4bbd-bc9c-45b1-9a96-b9cf3e342732","Type":"ContainerDied","Data":"6d8e0aec4253f9d89636b39a45f4ececcd3ae84e471acd157c7ad73f9e0f6c92"} Sep 07 13:21:05.142164 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.142113 2564 scope.go:117] "RemoveContainer" containerID="c530793caf05b230ae9eb580f24275f25f3807134c933b968d432358be5f7a09" Sep 07 13:21:05.143320 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.143286 2564 generic.go:356] "Generic (PLEG): container finished" podID="1621decd-bc38-4243-8f70-0873c25a339d" containerID="e2fdaa995bc99b719a83aa61aa2c1a4fbb078bf26d5edcb39ac4d922aa550f54" exitCode=0 Sep 07 13:21:05.143320 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.143322 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-57ttw" Sep 07 13:21:05.143499 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.143355 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-57ttw" event={"ID":"1621decd-bc38-4243-8f70-0873c25a339d","Type":"ContainerDied","Data":"e2fdaa995bc99b719a83aa61aa2c1a4fbb078bf26d5edcb39ac4d922aa550f54"} Sep 07 13:21:05.143499 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.143385 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-57ttw" event={"ID":"1621decd-bc38-4243-8f70-0873c25a339d","Type":"ContainerDied","Data":"a9046ba5a77e275b58bda96d61b20ab8ea1d5b76ba1ac016651a62bd89cd6774"} Sep 07 13:21:05.150228 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.150200 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz8jg\" (UniqueName: \"kubernetes.io/projected/32db4bbd-bc9c-45b1-9a96-b9cf3e342732-kube-api-access-qz8jg\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:21:05.150344 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.150231 2564 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/1621decd-bc38-4243-8f70-0873c25a339d-postgresdata\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:21:05.150344 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.150248 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fs2cg\" (UniqueName: \"kubernetes.io/projected/1621decd-bc38-4243-8f70-0873c25a339d-kube-api-access-fs2cg\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:21:05.150636 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.150470 2564 scope.go:117] "RemoveContainer" containerID="c530793caf05b230ae9eb580f24275f25f3807134c933b968d432358be5f7a09" Sep 07 13:21:05.150864 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:21:05.150835 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c530793caf05b230ae9eb580f24275f25f3807134c933b968d432358be5f7a09\": container with ID starting with c530793caf05b230ae9eb580f24275f25f3807134c933b968d432358be5f7a09 not found: ID does not exist" containerID="c530793caf05b230ae9eb580f24275f25f3807134c933b968d432358be5f7a09" Sep 07 13:21:05.150977 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.150869 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c530793caf05b230ae9eb580f24275f25f3807134c933b968d432358be5f7a09"} err="failed to get container status \"c530793caf05b230ae9eb580f24275f25f3807134c933b968d432358be5f7a09\": rpc error: code = NotFound desc = could not find container \"c530793caf05b230ae9eb580f24275f25f3807134c933b968d432358be5f7a09\": container with ID starting with c530793caf05b230ae9eb580f24275f25f3807134c933b968d432358be5f7a09 not found: ID does not exist" Sep 07 13:21:05.150977 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.150886 2564 scope.go:117] "RemoveContainer" containerID="e2fdaa995bc99b719a83aa61aa2c1a4fbb078bf26d5edcb39ac4d922aa550f54" Sep 07 13:21:05.158659 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.158639 2564 scope.go:117] "RemoveContainer" containerID="e2fdaa995bc99b719a83aa61aa2c1a4fbb078bf26d5edcb39ac4d922aa550f54" Sep 07 13:21:05.158910 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:21:05.158894 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2fdaa995bc99b719a83aa61aa2c1a4fbb078bf26d5edcb39ac4d922aa550f54\": container with ID starting with e2fdaa995bc99b719a83aa61aa2c1a4fbb078bf26d5edcb39ac4d922aa550f54 not found: ID does not exist" containerID="e2fdaa995bc99b719a83aa61aa2c1a4fbb078bf26d5edcb39ac4d922aa550f54" Sep 07 13:21:05.158972 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.158914 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2fdaa995bc99b719a83aa61aa2c1a4fbb078bf26d5edcb39ac4d922aa550f54"} err="failed to get container status \"e2fdaa995bc99b719a83aa61aa2c1a4fbb078bf26d5edcb39ac4d922aa550f54\": rpc error: code = NotFound desc = could not find container \"e2fdaa995bc99b719a83aa61aa2c1a4fbb078bf26d5edcb39ac4d922aa550f54\": container with ID starting with e2fdaa995bc99b719a83aa61aa2c1a4fbb078bf26d5edcb39ac4d922aa550f54 not found: ID does not exist" Sep 07 13:21:05.165655 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.165636 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-487ps"] Sep 07 13:21:05.170166 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.170148 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-487ps"] Sep 07 13:21:05.180756 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.180735 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-57ttw"] Sep 07 13:21:05.184758 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.184740 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-57ttw"] Sep 07 13:21:05.922609 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.922572 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1621decd-bc38-4243-8f70-0873c25a339d" path="/var/lib/kubelet/pods/1621decd-bc38-4243-8f70-0873c25a339d/volumes" Sep 07 13:21:05.922895 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:05.922883 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32db4bbd-bc9c-45b1-9a96-b9cf3e342732" path="/var/lib/kubelet/pods/32db4bbd-bc9c-45b1-9a96-b9cf3e342732/volumes" Sep 07 13:21:14.506612 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:14.506569 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Sep 07 13:21:15.086396 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:15.086342 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Sep 07 13:21:24.506356 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:24.506310 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Sep 07 13:21:24.508683 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:24.506441 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:21:25.086008 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:25.085963 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Sep 07 13:21:25.086215 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:25.086125 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:21:33.266579 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.266544 2564 generic.go:356] "Generic (PLEG): container finished" podID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerID="97b62e4dc281435ffd8984744b93bedcb640609d6338478eb9313106c34599e7" exitCode=137 Sep 07 13:21:33.267025 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.266624 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" event={"ID":"9728dd62-61bd-4385-9c96-5b8a63b2cef7","Type":"ContainerDied","Data":"97b62e4dc281435ffd8984744b93bedcb640609d6338478eb9313106c34599e7"} Sep 07 13:21:33.269284 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.269261 2564 generic.go:356] "Generic (PLEG): container finished" podID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerID="292d1811b5a1a3b45e7fd6a912a095b7012df61967887afaa41c4cec09beefc2" exitCode=137 Sep 07 13:21:33.269415 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.269309 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" event={"ID":"734b5f5a-5ce3-474c-89ad-81eadf32bc2e","Type":"ContainerDied","Data":"292d1811b5a1a3b45e7fd6a912a095b7012df61967887afaa41c4cec09beefc2"} Sep 07 13:21:33.444446 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.444421 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:21:33.469502 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.469470 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:21:33.598218 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.598123 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rszbs\" (UniqueName: \"kubernetes.io/projected/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-kube-api-access-rszbs\") pod \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\" (UID: \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\") " Sep 07 13:21:33.598218 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.598168 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-registry-tls\") pod \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\" (UID: \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\") " Sep 07 13:21:33.598218 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.598211 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9728dd62-61bd-4385-9c96-5b8a63b2cef7-registry-tls\") pod \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " Sep 07 13:21:33.598457 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.598319 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9728dd62-61bd-4385-9c96-5b8a63b2cef7-online-tls\") pod \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " Sep 07 13:21:33.598457 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.598365 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6qf8\" (UniqueName: \"kubernetes.io/projected/9728dd62-61bd-4385-9c96-5b8a63b2cef7-kube-api-access-j6qf8\") pod \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " Sep 07 13:21:33.598457 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.598416 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-feast-data\") pod \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\" (UID: \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\") " Sep 07 13:21:33.598457 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.598436 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9728dd62-61bd-4385-9c96-5b8a63b2cef7-feast-data\") pod \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\" (UID: \"9728dd62-61bd-4385-9c96-5b8a63b2cef7\") " Sep 07 13:21:33.598457 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.598454 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-online-tls\") pod \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\" (UID: \"734b5f5a-5ce3-474c-89ad-81eadf32bc2e\") " Sep 07 13:21:33.599221 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.599193 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9728dd62-61bd-4385-9c96-5b8a63b2cef7-feast-data" (OuterVolumeSpecName: "feast-data") pod "9728dd62-61bd-4385-9c96-5b8a63b2cef7" (UID: "9728dd62-61bd-4385-9c96-5b8a63b2cef7"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:21:33.600916 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.600887 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9728dd62-61bd-4385-9c96-5b8a63b2cef7-kube-api-access-j6qf8" (OuterVolumeSpecName: "kube-api-access-j6qf8") pod "9728dd62-61bd-4385-9c96-5b8a63b2cef7" (UID: "9728dd62-61bd-4385-9c96-5b8a63b2cef7"). InnerVolumeSpecName "kube-api-access-j6qf8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:21:33.601027 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.600951 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-kube-api-access-rszbs" (OuterVolumeSpecName: "kube-api-access-rszbs") pod "734b5f5a-5ce3-474c-89ad-81eadf32bc2e" (UID: "734b5f5a-5ce3-474c-89ad-81eadf32bc2e"). InnerVolumeSpecName "kube-api-access-rszbs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:21:33.601027 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.600950 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9728dd62-61bd-4385-9c96-5b8a63b2cef7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9728dd62-61bd-4385-9c96-5b8a63b2cef7" (UID: "9728dd62-61bd-4385-9c96-5b8a63b2cef7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:21:33.601027 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.600998 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "734b5f5a-5ce3-474c-89ad-81eadf32bc2e" (UID: "734b5f5a-5ce3-474c-89ad-81eadf32bc2e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:21:33.601171 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.601092 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9728dd62-61bd-4385-9c96-5b8a63b2cef7-online-tls" (OuterVolumeSpecName: "online-tls") pod "9728dd62-61bd-4385-9c96-5b8a63b2cef7" (UID: "9728dd62-61bd-4385-9c96-5b8a63b2cef7"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:21:33.601241 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.601223 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-online-tls" (OuterVolumeSpecName: "online-tls") pod "734b5f5a-5ce3-474c-89ad-81eadf32bc2e" (UID: "734b5f5a-5ce3-474c-89ad-81eadf32bc2e"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 13:21:33.607766 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.607745 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-feast-data" (OuterVolumeSpecName: "feast-data") pod "734b5f5a-5ce3-474c-89ad-81eadf32bc2e" (UID: "734b5f5a-5ce3-474c-89ad-81eadf32bc2e"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:21:33.699878 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.699831 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rszbs\" (UniqueName: \"kubernetes.io/projected/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-kube-api-access-rszbs\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:21:33.699878 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.699868 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-registry-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:21:33.699878 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.699878 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9728dd62-61bd-4385-9c96-5b8a63b2cef7-registry-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:21:33.700154 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.699907 2564 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9728dd62-61bd-4385-9c96-5b8a63b2cef7-online-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:21:33.700154 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.699917 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6qf8\" (UniqueName: \"kubernetes.io/projected/9728dd62-61bd-4385-9c96-5b8a63b2cef7-kube-api-access-j6qf8\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:21:33.700154 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.699925 2564 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-feast-data\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:21:33.700154 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.699933 2564 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9728dd62-61bd-4385-9c96-5b8a63b2cef7-feast-data\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:21:33.700154 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:33.699941 2564 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/734b5f5a-5ce3-474c-89ad-81eadf32bc2e-online-tls\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:21:34.275360 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:34.275327 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" Sep 07 13:21:34.275834 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:34.275364 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx" event={"ID":"9728dd62-61bd-4385-9c96-5b8a63b2cef7","Type":"ContainerDied","Data":"3fba784e2e6f902f6379f83f04f9684996c571b7a979efb3fcb020e396084716"} Sep 07 13:21:34.275834 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:34.275426 2564 scope.go:117] "RemoveContainer" containerID="d667ef1d157723c1866086c262e0838f6dfe8f20ade68c87846f025138486dc0" Sep 07 13:21:34.278014 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:34.277995 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" Sep 07 13:21:34.278115 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:34.277993 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm" event={"ID":"734b5f5a-5ce3-474c-89ad-81eadf32bc2e","Type":"ContainerDied","Data":"fc06119569a61459a982af4303f48305a82028aa5f1c98f45c074110db19d651"} Sep 07 13:21:34.283891 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:34.283872 2564 scope.go:117] "RemoveContainer" containerID="97b62e4dc281435ffd8984744b93bedcb640609d6338478eb9313106c34599e7" Sep 07 13:21:34.291933 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:34.291908 2564 scope.go:117] "RemoveContainer" containerID="f28130cf55d9eaac8ec7db257b2445657f429461bf8d633177fe065b2e0615ce" Sep 07 13:21:34.299565 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:34.299544 2564 scope.go:117] "RemoveContainer" containerID="3b19fd94ebd6e18473e279ec4ca012c4d4c94da98d3758f69203ab89de8dd549" Sep 07 13:21:34.307174 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:34.307150 2564 scope.go:117] "RemoveContainer" containerID="6981bfe74f12cdb0847074aa58bf95fa188dd6ff7855add818854f1dbb263dbc" Sep 07 13:21:34.315008 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:34.314974 2564 scope.go:117] "RemoveContainer" containerID="292d1811b5a1a3b45e7fd6a912a095b7012df61967887afaa41c4cec09beefc2" Sep 07 13:21:34.316458 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:34.316439 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm"] Sep 07 13:21:34.320819 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:34.320798 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-f9c7bbf4f-dsdjm"] Sep 07 13:21:34.323844 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:34.323825 2564 scope.go:117] "RemoveContainer" containerID="c02ccc68565cc6b7a6ccda92c36bc0a1b633a8381804c3e09a87da8d1c277ec0" Sep 07 13:21:34.331270 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:34.331251 2564 scope.go:117] "RemoveContainer" containerID="2fbea6d52d9d2d9016b689d124c20e285ba1294d0fea495efa56d74b34173609" Sep 07 13:21:34.337551 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:34.337515 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx"] Sep 07 13:21:34.340130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:34.340099 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-78b9b4767d-fnnvx"] Sep 07 13:21:35.921163 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:35.921130 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" path="/var/lib/kubelet/pods/734b5f5a-5ce3-474c-89ad-81eadf32bc2e/volumes" Sep 07 13:21:35.921638 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:21:35.921624 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" path="/var/lib/kubelet/pods/9728dd62-61bd-4385-9c96-5b8a63b2cef7/volumes" Sep 07 13:32:45.316684 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.316646 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gzls5/must-gather-l2btk"] Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.316937 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50f7a4f8-db11-4632-bd06-4e689939d4df" containerName="manager" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.316948 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="50f7a4f8-db11-4632-bd06-4e689939d4df" containerName="manager" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.316961 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="feast-apply" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.316967 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="feast-apply" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.316977 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="feast-init" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.316983 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="feast-init" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.316989 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="registry" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.316993 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="registry" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317000 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="online" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317005 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="online" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317013 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1621decd-bc38-4243-8f70-0873c25a339d" containerName="postgres" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317017 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1621decd-bc38-4243-8f70-0873c25a339d" containerName="postgres" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317024 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="registry" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317028 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="registry" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317034 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32db4bbd-bc9c-45b1-9a96-b9cf3e342732" containerName="redis" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317039 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="32db4bbd-bc9c-45b1-9a96-b9cf3e342732" containerName="redis" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317063 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="feast-init" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317069 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="feast-init" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317074 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="feast-apply" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317080 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="feast-apply" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317088 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="online" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317093 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="online" Sep 07 13:32:45.317130 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317140 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="32db4bbd-bc9c-45b1-9a96-b9cf3e342732" containerName="redis" Sep 07 13:32:45.317802 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317147 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1621decd-bc38-4243-8f70-0873c25a339d" containerName="postgres" Sep 07 13:32:45.317802 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317153 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="online" Sep 07 13:32:45.317802 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317161 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="online" Sep 07 13:32:45.317802 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317168 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="734b5f5a-5ce3-474c-89ad-81eadf32bc2e" containerName="registry" Sep 07 13:32:45.317802 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317173 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9728dd62-61bd-4385-9c96-5b8a63b2cef7" containerName="registry" Sep 07 13:32:45.317802 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.317178 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="50f7a4f8-db11-4632-bd06-4e689939d4df" containerName="manager" Sep 07 13:32:45.320047 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.320028 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gzls5/must-gather-l2btk" Sep 07 13:32:45.324842 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.324805 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gzls5\"/\"default-dockercfg-frr7h\"" Sep 07 13:32:45.325583 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.325562 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gzls5\"/\"openshift-service-ca.crt\"" Sep 07 13:32:45.325689 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.325602 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gzls5\"/\"kube-root-ca.crt\"" Sep 07 13:32:45.337846 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.337819 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zbff\" (UniqueName: \"kubernetes.io/projected/5cbdfd88-ec28-4a80-acbe-707b6de89a2b-kube-api-access-5zbff\") pod \"must-gather-l2btk\" (UID: \"5cbdfd88-ec28-4a80-acbe-707b6de89a2b\") " pod="openshift-must-gather-gzls5/must-gather-l2btk" Sep 07 13:32:45.337946 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.337905 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5cbdfd88-ec28-4a80-acbe-707b6de89a2b-must-gather-output\") pod \"must-gather-l2btk\" (UID: \"5cbdfd88-ec28-4a80-acbe-707b6de89a2b\") " pod="openshift-must-gather-gzls5/must-gather-l2btk" Sep 07 13:32:45.348976 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.348951 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gzls5/must-gather-l2btk"] Sep 07 13:32:45.438873 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.438835 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5cbdfd88-ec28-4a80-acbe-707b6de89a2b-must-gather-output\") pod \"must-gather-l2btk\" (UID: \"5cbdfd88-ec28-4a80-acbe-707b6de89a2b\") " pod="openshift-must-gather-gzls5/must-gather-l2btk" Sep 07 13:32:45.439094 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.438897 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zbff\" (UniqueName: \"kubernetes.io/projected/5cbdfd88-ec28-4a80-acbe-707b6de89a2b-kube-api-access-5zbff\") pod \"must-gather-l2btk\" (UID: \"5cbdfd88-ec28-4a80-acbe-707b6de89a2b\") " pod="openshift-must-gather-gzls5/must-gather-l2btk" Sep 07 13:32:45.439291 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.439268 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5cbdfd88-ec28-4a80-acbe-707b6de89a2b-must-gather-output\") pod \"must-gather-l2btk\" (UID: \"5cbdfd88-ec28-4a80-acbe-707b6de89a2b\") " pod="openshift-must-gather-gzls5/must-gather-l2btk" Sep 07 13:32:45.450152 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.450120 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zbff\" (UniqueName: \"kubernetes.io/projected/5cbdfd88-ec28-4a80-acbe-707b6de89a2b-kube-api-access-5zbff\") pod \"must-gather-l2btk\" (UID: \"5cbdfd88-ec28-4a80-acbe-707b6de89a2b\") " pod="openshift-must-gather-gzls5/must-gather-l2btk" Sep 07 13:32:45.641662 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.641555 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gzls5/must-gather-l2btk" Sep 07 13:32:45.910629 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.910599 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gzls5/must-gather-l2btk"] Sep 07 13:32:45.913314 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:32:45.913282 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5cbdfd88_ec28_4a80_acbe_707b6de89a2b.slice/crio-063ce74fe4e1b9dd62445aa62741e490403685e8ee8ba6a958cbc8bf902c2321 WatchSource:0}: Error finding container 063ce74fe4e1b9dd62445aa62741e490403685e8ee8ba6a958cbc8bf902c2321: Status 404 returned error can't find the container with id 063ce74fe4e1b9dd62445aa62741e490403685e8ee8ba6a958cbc8bf902c2321 Sep 07 13:32:45.915202 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:45.915181 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 13:32:46.849157 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:46.849068 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gzls5/must-gather-l2btk" event={"ID":"5cbdfd88-ec28-4a80-acbe-707b6de89a2b","Type":"ContainerStarted","Data":"063ce74fe4e1b9dd62445aa62741e490403685e8ee8ba6a958cbc8bf902c2321"} Sep 07 13:32:52.880277 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:52.880241 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gzls5/must-gather-l2btk" event={"ID":"5cbdfd88-ec28-4a80-acbe-707b6de89a2b","Type":"ContainerStarted","Data":"5ee7b615959c5f1b76299ed1b32b068e6e47576df4e1feb0a70f5a8da57d8ba9"} Sep 07 13:32:52.880803 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:32:52.880286 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gzls5/must-gather-l2btk" event={"ID":"5cbdfd88-ec28-4a80-acbe-707b6de89a2b","Type":"ContainerStarted","Data":"6e798d9b8d38a30d2e4c7d17d4d24980759c0fbea7f9a267486f48eab8973de6"} Sep 07 13:33:01.917701 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:01.917662 2564 generic.go:356] "Generic (PLEG): container finished" podID="5cbdfd88-ec28-4a80-acbe-707b6de89a2b" containerID="6e798d9b8d38a30d2e4c7d17d4d24980759c0fbea7f9a267486f48eab8973de6" exitCode=0 Sep 07 13:33:01.927038 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:01.927005 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gzls5/must-gather-l2btk" event={"ID":"5cbdfd88-ec28-4a80-acbe-707b6de89a2b","Type":"ContainerDied","Data":"6e798d9b8d38a30d2e4c7d17d4d24980759c0fbea7f9a267486f48eab8973de6"} Sep 07 13:33:01.927321 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:01.927305 2564 scope.go:117] "RemoveContainer" containerID="6e798d9b8d38a30d2e4c7d17d4d24980759c0fbea7f9a267486f48eab8973de6" Sep 07 13:33:02.833928 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:02.833896 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-gzls5_must-gather-l2btk_5cbdfd88-ec28-4a80-acbe-707b6de89a2b/gather/0.log" Sep 07 13:33:06.048290 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:06.048259 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k8jtg_cefbef21-0cc5-4d7a-8dc3-e706bef2fba8/global-pull-secret-syncer/0.log" Sep 07 13:33:06.270083 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:06.270040 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tj9nj_3c09e344-284e-410e-8829-3b0bf6d35c93/konnectivity-agent/0.log" Sep 07 13:33:06.314100 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:06.313998 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-45.ec2.internal_e4dcf97b5b5636889d8eabaae498c349/haproxy/0.log" Sep 07 13:33:08.156007 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.155970 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-gzls5/must-gather-l2btk"] Sep 07 13:33:08.156526 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.156143 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-gzls5/must-gather-l2btk" podUID="5cbdfd88-ec28-4a80-acbe-707b6de89a2b" containerName="copy" containerID="cri-o://5ee7b615959c5f1b76299ed1b32b068e6e47576df4e1feb0a70f5a8da57d8ba9" gracePeriod=2 Sep 07 13:33:08.160284 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.160089 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-gzls5/must-gather-l2btk"] Sep 07 13:33:08.387723 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.387700 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-gzls5_must-gather-l2btk_5cbdfd88-ec28-4a80-acbe-707b6de89a2b/copy/0.log" Sep 07 13:33:08.390402 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.390382 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gzls5/must-gather-l2btk" Sep 07 13:33:08.392496 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:33:08.392468 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-l2btk\" is forbidden: User \"system:node:ip-10-0-139-45.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-gzls5\": no relationship found between node 'ip-10-0-139-45.ec2.internal' and this object" podUID="5cbdfd88-ec28-4a80-acbe-707b6de89a2b" pod="openshift-must-gather-gzls5/must-gather-l2btk" Sep 07 13:33:08.534072 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.534010 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5cbdfd88-ec28-4a80-acbe-707b6de89a2b-must-gather-output\") pod \"5cbdfd88-ec28-4a80-acbe-707b6de89a2b\" (UID: \"5cbdfd88-ec28-4a80-acbe-707b6de89a2b\") " Sep 07 13:33:08.534273 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.534093 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zbff\" (UniqueName: \"kubernetes.io/projected/5cbdfd88-ec28-4a80-acbe-707b6de89a2b-kube-api-access-5zbff\") pod \"5cbdfd88-ec28-4a80-acbe-707b6de89a2b\" (UID: \"5cbdfd88-ec28-4a80-acbe-707b6de89a2b\") " Sep 07 13:33:08.534453 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.534424 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cbdfd88-ec28-4a80-acbe-707b6de89a2b-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "5cbdfd88-ec28-4a80-acbe-707b6de89a2b" (UID: "5cbdfd88-ec28-4a80-acbe-707b6de89a2b"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 13:33:08.536283 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.536255 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cbdfd88-ec28-4a80-acbe-707b6de89a2b-kube-api-access-5zbff" (OuterVolumeSpecName: "kube-api-access-5zbff") pod "5cbdfd88-ec28-4a80-acbe-707b6de89a2b" (UID: "5cbdfd88-ec28-4a80-acbe-707b6de89a2b"). InnerVolumeSpecName "kube-api-access-5zbff". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 13:33:08.634886 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.634847 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zbff\" (UniqueName: \"kubernetes.io/projected/5cbdfd88-ec28-4a80-acbe-707b6de89a2b-kube-api-access-5zbff\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:33:08.634886 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.634875 2564 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5cbdfd88-ec28-4a80-acbe-707b6de89a2b-must-gather-output\") on node \"ip-10-0-139-45.ec2.internal\" DevicePath \"\"" Sep 07 13:33:08.945043 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.944960 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-gzls5_must-gather-l2btk_5cbdfd88-ec28-4a80-acbe-707b6de89a2b/copy/0.log" Sep 07 13:33:08.945363 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.945341 2564 generic.go:356] "Generic (PLEG): container finished" podID="5cbdfd88-ec28-4a80-acbe-707b6de89a2b" containerID="5ee7b615959c5f1b76299ed1b32b068e6e47576df4e1feb0a70f5a8da57d8ba9" exitCode=143 Sep 07 13:33:08.945421 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.945388 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gzls5/must-gather-l2btk" Sep 07 13:33:08.945482 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.945393 2564 scope.go:117] "RemoveContainer" containerID="5ee7b615959c5f1b76299ed1b32b068e6e47576df4e1feb0a70f5a8da57d8ba9" Sep 07 13:33:08.947686 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:33:08.947651 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-l2btk\" is forbidden: User \"system:node:ip-10-0-139-45.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-gzls5\": no relationship found between node 'ip-10-0-139-45.ec2.internal' and this object" podUID="5cbdfd88-ec28-4a80-acbe-707b6de89a2b" pod="openshift-must-gather-gzls5/must-gather-l2btk" Sep 07 13:33:08.957856 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.957829 2564 scope.go:117] "RemoveContainer" containerID="6e798d9b8d38a30d2e4c7d17d4d24980759c0fbea7f9a267486f48eab8973de6" Sep 07 13:33:08.959229 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:33:08.959197 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-l2btk\" is forbidden: User \"system:node:ip-10-0-139-45.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-gzls5\": no relationship found between node 'ip-10-0-139-45.ec2.internal' and this object" podUID="5cbdfd88-ec28-4a80-acbe-707b6de89a2b" pod="openshift-must-gather-gzls5/must-gather-l2btk" Sep 07 13:33:08.969068 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.969006 2564 scope.go:117] "RemoveContainer" containerID="5ee7b615959c5f1b76299ed1b32b068e6e47576df4e1feb0a70f5a8da57d8ba9" Sep 07 13:33:08.969527 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:33:08.969497 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ee7b615959c5f1b76299ed1b32b068e6e47576df4e1feb0a70f5a8da57d8ba9\": container with ID starting with 5ee7b615959c5f1b76299ed1b32b068e6e47576df4e1feb0a70f5a8da57d8ba9 not found: ID does not exist" containerID="5ee7b615959c5f1b76299ed1b32b068e6e47576df4e1feb0a70f5a8da57d8ba9" Sep 07 13:33:08.969592 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.969544 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ee7b615959c5f1b76299ed1b32b068e6e47576df4e1feb0a70f5a8da57d8ba9"} err="failed to get container status \"5ee7b615959c5f1b76299ed1b32b068e6e47576df4e1feb0a70f5a8da57d8ba9\": rpc error: code = NotFound desc = could not find container \"5ee7b615959c5f1b76299ed1b32b068e6e47576df4e1feb0a70f5a8da57d8ba9\": container with ID starting with 5ee7b615959c5f1b76299ed1b32b068e6e47576df4e1feb0a70f5a8da57d8ba9 not found: ID does not exist" Sep 07 13:33:08.969592 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.969573 2564 scope.go:117] "RemoveContainer" containerID="6e798d9b8d38a30d2e4c7d17d4d24980759c0fbea7f9a267486f48eab8973de6" Sep 07 13:33:08.969849 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:33:08.969830 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e798d9b8d38a30d2e4c7d17d4d24980759c0fbea7f9a267486f48eab8973de6\": container with ID starting with 6e798d9b8d38a30d2e4c7d17d4d24980759c0fbea7f9a267486f48eab8973de6 not found: ID does not exist" containerID="6e798d9b8d38a30d2e4c7d17d4d24980759c0fbea7f9a267486f48eab8973de6" Sep 07 13:33:08.969897 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:08.969855 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e798d9b8d38a30d2e4c7d17d4d24980759c0fbea7f9a267486f48eab8973de6"} err="failed to get container status \"6e798d9b8d38a30d2e4c7d17d4d24980759c0fbea7f9a267486f48eab8973de6\": rpc error: code = NotFound desc = could not find container \"6e798d9b8d38a30d2e4c7d17d4d24980759c0fbea7f9a267486f48eab8973de6\": container with ID starting with 6e798d9b8d38a30d2e4c7d17d4d24980759c0fbea7f9a267486f48eab8973de6 not found: ID does not exist" Sep 07 13:33:09.920104 ip-10-0-139-45 kubenswrapper[2564]: E0907 13:33:09.920066 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-l2btk\" is forbidden: User \"system:node:ip-10-0-139-45.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-gzls5\": no relationship found between node 'ip-10-0-139-45.ec2.internal' and this object" podUID="5cbdfd88-ec28-4a80-acbe-707b6de89a2b" pod="openshift-must-gather-gzls5/must-gather-l2btk" Sep 07 13:33:09.921529 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:09.921509 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cbdfd88-ec28-4a80-acbe-707b6de89a2b" path="/var/lib/kubelet/pods/5cbdfd88-ec28-4a80-acbe-707b6de89a2b/volumes" Sep 07 13:33:10.162429 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:10.162388 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f88d5418-255b-48e8-822f-8cb0ff7ee119/alertmanager/0.log" Sep 07 13:33:10.218729 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:10.218653 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f88d5418-255b-48e8-822f-8cb0ff7ee119/config-reloader/0.log" Sep 07 13:33:10.300366 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:10.300336 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f88d5418-255b-48e8-822f-8cb0ff7ee119/kube-rbac-proxy-web/0.log" Sep 07 13:33:10.330948 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:10.330916 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f88d5418-255b-48e8-822f-8cb0ff7ee119/kube-rbac-proxy/0.log" Sep 07 13:33:10.356002 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:10.355967 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f88d5418-255b-48e8-822f-8cb0ff7ee119/kube-rbac-proxy-metric/0.log" Sep 07 13:33:10.380179 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:10.380148 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f88d5418-255b-48e8-822f-8cb0ff7ee119/prom-label-proxy/0.log" Sep 07 13:33:10.405444 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:10.405405 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f88d5418-255b-48e8-822f-8cb0ff7ee119/init-config-reloader/0.log" Sep 07 13:33:10.562885 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:10.562829 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-th7sq_723b3e1d-a3f4-491f-b45e-f20b1972b28c/monitoring-plugin/0.log" Sep 07 13:33:10.675023 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:10.674988 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7gqks_902a96ad-535c-44a0-b497-56b5f03ff2aa/node-exporter/0.log" Sep 07 13:33:10.699712 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:10.699680 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7gqks_902a96ad-535c-44a0-b497-56b5f03ff2aa/kube-rbac-proxy/0.log" Sep 07 13:33:10.724667 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:10.724642 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7gqks_902a96ad-535c-44a0-b497-56b5f03ff2aa/init-textfile/0.log" Sep 07 13:33:10.838667 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:10.838590 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-frjrr_ce4d061f-a135-4f03-8326-8eccb67ce345/kube-rbac-proxy-main/0.log" Sep 07 13:33:10.864988 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:10.864963 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-frjrr_ce4d061f-a135-4f03-8326-8eccb67ce345/kube-rbac-proxy-self/0.log" Sep 07 13:33:10.902743 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:10.902714 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-frjrr_ce4d061f-a135-4f03-8326-8eccb67ce345/openshift-state-metrics/0.log" Sep 07 13:33:11.228408 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:11.228379 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-lj5t6_a0aca2db-7f85-4df0-8440-ad3e9767d3c9/prometheus-operator/0.log" Sep 07 13:33:11.249249 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:11.249218 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-lj5t6_a0aca2db-7f85-4df0-8440-ad3e9767d3c9/kube-rbac-proxy/0.log" Sep 07 13:33:11.275885 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:11.275857 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-nkfjl_29d0b5ca-7cd9-49fd-b14f-4f16ffbe67bc/prometheus-operator-admission-webhook/0.log" Sep 07 13:33:11.313515 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:11.313484 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85fd5c4f45-mc5hj_ce3bfbce-e613-4016-ae00-a3b49a64e500/telemeter-client/0.log" Sep 07 13:33:11.338955 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:11.338918 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85fd5c4f45-mc5hj_ce3bfbce-e613-4016-ae00-a3b49a64e500/reload/0.log" Sep 07 13:33:11.364168 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:11.364137 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85fd5c4f45-mc5hj_ce3bfbce-e613-4016-ae00-a3b49a64e500/kube-rbac-proxy/0.log" Sep 07 13:33:12.891625 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.891586 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9"] Sep 07 13:33:12.892013 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.891936 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cbdfd88-ec28-4a80-acbe-707b6de89a2b" containerName="gather" Sep 07 13:33:12.892013 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.891950 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cbdfd88-ec28-4a80-acbe-707b6de89a2b" containerName="gather" Sep 07 13:33:12.892013 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.891959 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cbdfd88-ec28-4a80-acbe-707b6de89a2b" containerName="copy" Sep 07 13:33:12.892013 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.891965 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cbdfd88-ec28-4a80-acbe-707b6de89a2b" containerName="copy" Sep 07 13:33:12.892172 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.892019 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cbdfd88-ec28-4a80-acbe-707b6de89a2b" containerName="gather" Sep 07 13:33:12.892172 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.892034 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cbdfd88-ec28-4a80-acbe-707b6de89a2b" containerName="copy" Sep 07 13:33:12.896895 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.896874 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:12.899670 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.899640 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fd6hd\"/\"kube-root-ca.crt\"" Sep 07 13:33:12.899803 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.899640 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fd6hd\"/\"openshift-service-ca.crt\"" Sep 07 13:33:12.900503 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.900484 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fd6hd\"/\"default-dockercfg-rpvlg\"" Sep 07 13:33:12.908107 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.908084 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9"] Sep 07 13:33:12.975167 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.975131 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cf03a086-87d7-40f8-92d1-c2d48e3989f6-proc\") pod \"perf-node-gather-daemonset-hq2v9\" (UID: \"cf03a086-87d7-40f8-92d1-c2d48e3989f6\") " pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:12.975370 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.975185 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql6bg\" (UniqueName: \"kubernetes.io/projected/cf03a086-87d7-40f8-92d1-c2d48e3989f6-kube-api-access-ql6bg\") pod \"perf-node-gather-daemonset-hq2v9\" (UID: \"cf03a086-87d7-40f8-92d1-c2d48e3989f6\") " pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:12.975370 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.975257 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf03a086-87d7-40f8-92d1-c2d48e3989f6-sys\") pod \"perf-node-gather-daemonset-hq2v9\" (UID: \"cf03a086-87d7-40f8-92d1-c2d48e3989f6\") " pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:12.975370 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.975291 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf03a086-87d7-40f8-92d1-c2d48e3989f6-lib-modules\") pod \"perf-node-gather-daemonset-hq2v9\" (UID: \"cf03a086-87d7-40f8-92d1-c2d48e3989f6\") " pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:12.975370 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:12.975323 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cf03a086-87d7-40f8-92d1-c2d48e3989f6-podres\") pod \"perf-node-gather-daemonset-hq2v9\" (UID: \"cf03a086-87d7-40f8-92d1-c2d48e3989f6\") " pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:13.076517 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.076472 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cf03a086-87d7-40f8-92d1-c2d48e3989f6-proc\") pod \"perf-node-gather-daemonset-hq2v9\" (UID: \"cf03a086-87d7-40f8-92d1-c2d48e3989f6\") " pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:13.076728 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.076530 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql6bg\" (UniqueName: \"kubernetes.io/projected/cf03a086-87d7-40f8-92d1-c2d48e3989f6-kube-api-access-ql6bg\") pod \"perf-node-gather-daemonset-hq2v9\" (UID: \"cf03a086-87d7-40f8-92d1-c2d48e3989f6\") " pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:13.076728 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.076550 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf03a086-87d7-40f8-92d1-c2d48e3989f6-sys\") pod \"perf-node-gather-daemonset-hq2v9\" (UID: \"cf03a086-87d7-40f8-92d1-c2d48e3989f6\") " pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:13.076728 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.076567 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf03a086-87d7-40f8-92d1-c2d48e3989f6-lib-modules\") pod \"perf-node-gather-daemonset-hq2v9\" (UID: \"cf03a086-87d7-40f8-92d1-c2d48e3989f6\") " pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:13.076728 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.076587 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cf03a086-87d7-40f8-92d1-c2d48e3989f6-podres\") pod \"perf-node-gather-daemonset-hq2v9\" (UID: \"cf03a086-87d7-40f8-92d1-c2d48e3989f6\") " pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:13.076728 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.076613 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cf03a086-87d7-40f8-92d1-c2d48e3989f6-proc\") pod \"perf-node-gather-daemonset-hq2v9\" (UID: \"cf03a086-87d7-40f8-92d1-c2d48e3989f6\") " pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:13.076728 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.076656 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf03a086-87d7-40f8-92d1-c2d48e3989f6-sys\") pod \"perf-node-gather-daemonset-hq2v9\" (UID: \"cf03a086-87d7-40f8-92d1-c2d48e3989f6\") " pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:13.077021 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.076738 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cf03a086-87d7-40f8-92d1-c2d48e3989f6-podres\") pod \"perf-node-gather-daemonset-hq2v9\" (UID: \"cf03a086-87d7-40f8-92d1-c2d48e3989f6\") " pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:13.077021 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.076740 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf03a086-87d7-40f8-92d1-c2d48e3989f6-lib-modules\") pod \"perf-node-gather-daemonset-hq2v9\" (UID: \"cf03a086-87d7-40f8-92d1-c2d48e3989f6\") " pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:13.087213 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.087186 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql6bg\" (UniqueName: \"kubernetes.io/projected/cf03a086-87d7-40f8-92d1-c2d48e3989f6-kube-api-access-ql6bg\") pod \"perf-node-gather-daemonset-hq2v9\" (UID: \"cf03a086-87d7-40f8-92d1-c2d48e3989f6\") " pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:13.206214 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.206115 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:13.479452 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.479418 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9"] Sep 07 13:33:13.482880 ip-10-0-139-45 kubenswrapper[2564]: W0907 13:33:13.482855 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcf03a086_87d7_40f8_92d1_c2d48e3989f6.slice/crio-b84f9ac38e66d706cfefb1e3938c50e2e2e3d085d557847bb24aaf502b2d94e5 WatchSource:0}: Error finding container b84f9ac38e66d706cfefb1e3938c50e2e2e3d085d557847bb24aaf502b2d94e5: Status 404 returned error can't find the container with id b84f9ac38e66d706cfefb1e3938c50e2e2e3d085d557847bb24aaf502b2d94e5 Sep 07 13:33:13.514687 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.514659 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d99c97c9f-5p8gn_2e29cc86-2a68-4f98-93b8-f8ee7296966a/console/0.log" Sep 07 13:33:13.551179 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.551154 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-zzch6_2458852c-2282-4634-9422-ff5843ccfb68/download-server/0.log" Sep 07 13:33:13.967589 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.967494 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" event={"ID":"cf03a086-87d7-40f8-92d1-c2d48e3989f6","Type":"ContainerStarted","Data":"87460ed362f32ffa6c1f40db363847083483b29e807cecc1a2a51b2a428d085f"} Sep 07 13:33:13.967589 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.967557 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" event={"ID":"cf03a086-87d7-40f8-92d1-c2d48e3989f6","Type":"ContainerStarted","Data":"b84f9ac38e66d706cfefb1e3938c50e2e2e3d085d557847bb24aaf502b2d94e5"} Sep 07 13:33:13.968045 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.967599 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:13.986481 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:13.986423 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" podStartSLOduration=1.986407705 podStartE2EDuration="1.986407705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 13:33:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 13:33:13.985548201 +0000 UTC m=+1954.581052817" watchObservedRunningTime="2026-09-07 13:33:13.986407705 +0000 UTC m=+1954.581912320" Sep 07 13:33:14.033809 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:14.033784 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-fwckf_2313840a-af08-4019-a75c-01db10b0df66/volume-data-source-validator/0.log" Sep 07 13:33:14.833481 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:14.833449 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-74wz2_66d7fa5d-e887-46a9-b527-a7a8398e4786/dns/0.log" Sep 07 13:33:14.864668 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:14.864640 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-74wz2_66d7fa5d-e887-46a9-b527-a7a8398e4786/kube-rbac-proxy/0.log" Sep 07 13:33:15.042031 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:15.041998 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fcd4k_912823fa-6c24-493a-8640-89b921cdf2f2/dns-node-resolver/0.log" Sep 07 13:33:15.527549 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:15.527519 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hrgm6_ec3637df-91b4-4cbe-8d8b-7378b2c12a4f/node-ca/0.log" Sep 07 13:33:16.293997 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:16.293969 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74489cdf66-fv7gs_790dc1f9-f44a-4c9d-a081-8de58dd6d08e/router/0.log" Sep 07 13:33:16.630923 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:16.630846 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6gpjx_3f13b2c1-d6fb-4722-b60e-4b940d617447/serve-healthcheck-canary/0.log" Sep 07 13:33:17.271204 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:17.271176 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mkg7m_845ff0c5-eab8-4c81-abca-19f574c1dcdb/kube-rbac-proxy/0.log" Sep 07 13:33:17.294741 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:17.294717 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mkg7m_845ff0c5-eab8-4c81-abca-19f574c1dcdb/exporter/0.log" Sep 07 13:33:17.317796 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:17.317761 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mkg7m_845ff0c5-eab8-4c81-abca-19f574c1dcdb/extractor/0.log" Sep 07 13:33:19.981095 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:19.981047 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fd6hd/perf-node-gather-daemonset-hq2v9" Sep 07 13:33:22.165389 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:22.165306 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-2qm2w_c7991f8f-dd41-437e-8d24-8d2b3c756304/kube-storage-version-migrator-operator/1.log" Sep 07 13:33:22.166368 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:22.166347 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-2qm2w_c7991f8f-dd41-437e-8d24-8d2b3c756304/kube-storage-version-migrator-operator/0.log" Sep 07 13:33:23.116390 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:23.116360 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-8rdps_84352554-8f00-4dfe-8a59-c14dc986c2dd/kube-multus/0.log" Sep 07 13:33:23.139145 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:23.139109 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4p6gh_568ca0aa-04bf-46b4-b3d1-38e935c0e5d8/kube-multus-additional-cni-plugins/0.log" Sep 07 13:33:23.160595 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:23.160568 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4p6gh_568ca0aa-04bf-46b4-b3d1-38e935c0e5d8/egress-router-binary-copy/0.log" Sep 07 13:33:23.184328 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:23.184304 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4p6gh_568ca0aa-04bf-46b4-b3d1-38e935c0e5d8/cni-plugins/0.log" Sep 07 13:33:23.206266 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:23.206233 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4p6gh_568ca0aa-04bf-46b4-b3d1-38e935c0e5d8/bond-cni-plugin/0.log" Sep 07 13:33:23.226839 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:23.226809 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4p6gh_568ca0aa-04bf-46b4-b3d1-38e935c0e5d8/routeoverride-cni/0.log" Sep 07 13:33:23.249463 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:23.249441 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4p6gh_568ca0aa-04bf-46b4-b3d1-38e935c0e5d8/whereabouts-cni-bincopy/0.log" Sep 07 13:33:23.269523 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:23.269498 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4p6gh_568ca0aa-04bf-46b4-b3d1-38e935c0e5d8/whereabouts-cni/0.log" Sep 07 13:33:23.732106 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:23.732075 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-f5cpd_5fbd07ff-dbb9-46f0-904d-d21419e31373/network-metrics-daemon/0.log" Sep 07 13:33:23.754722 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:23.754693 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-f5cpd_5fbd07ff-dbb9-46f0-904d-d21419e31373/kube-rbac-proxy/0.log" Sep 07 13:33:24.934081 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:24.934029 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wxbzw_0fc4604b-fd16-466e-8a28-e77878ba81cd/ovn-controller/0.log" Sep 07 13:33:24.960089 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:24.960047 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wxbzw_0fc4604b-fd16-466e-8a28-e77878ba81cd/ovn-acl-logging/0.log" Sep 07 13:33:24.987409 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:24.987383 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wxbzw_0fc4604b-fd16-466e-8a28-e77878ba81cd/kube-rbac-proxy-node/0.log" Sep 07 13:33:25.014797 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:25.014763 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wxbzw_0fc4604b-fd16-466e-8a28-e77878ba81cd/kube-rbac-proxy-ovn-metrics/0.log" Sep 07 13:33:25.037897 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:25.037867 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wxbzw_0fc4604b-fd16-466e-8a28-e77878ba81cd/northd/0.log" Sep 07 13:33:25.077274 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:25.077245 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wxbzw_0fc4604b-fd16-466e-8a28-e77878ba81cd/nbdb/0.log" Sep 07 13:33:25.099648 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:25.099619 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wxbzw_0fc4604b-fd16-466e-8a28-e77878ba81cd/sbdb/0.log" Sep 07 13:33:25.142823 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:25.142773 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wxbzw_0fc4604b-fd16-466e-8a28-e77878ba81cd/ovnkube-controller/0.log" Sep 07 13:33:26.293388 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:26.293356 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-plvr8_e754de95-041a-497a-bee1-8ca095b326f4/network-check-target-container/0.log" Sep 07 13:33:27.267249 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:27.267216 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-w27xg_8ed5b1e0-eb99-433d-9df9-43019c49d31a/iptables-alerter/0.log" Sep 07 13:33:27.943907 ip-10-0-139-45 kubenswrapper[2564]: I0907 13:33:27.943872 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-9rrnt_a85534bb-a70d-4c02-a132-b56a2faf2c47/tuned/0.log"