Sep 17 07:06:06.123639 ip-10-0-142-90 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 17 07:06:06.123650 ip-10-0-142-90 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 17 07:06:06.123657 ip-10-0-142-90 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 17 07:06:06.123813 ip-10-0-142-90 systemd[1]: Failed to start Kubernetes Kubelet. Sep 17 07:06:16.270656 ip-10-0-142-90 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 17 07:06:16.270678 ip-10-0-142-90 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 21b42b86a0164d909f9b2ed6093aaeca -- Sep 17 07:08:05.985712 ip-10-0-142-90 systemd[1]: Starting Kubernetes Kubelet... Sep 17 07:08:06.419651 ip-10-0-142-90 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 07:08:06.419651 ip-10-0-142-90 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 17 07:08:06.419651 ip-10-0-142-90 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 07:08:06.419651 ip-10-0-142-90 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 17 07:08:06.419651 ip-10-0-142-90 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 07:08:06.422195 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.422102 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 17 07:08:06.426416 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426386 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:08:06.426416 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426416 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426419 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426423 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426426 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426430 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426433 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426436 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426439 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426441 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426444 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426454 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426457 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426460 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426462 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426465 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426468 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426471 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426473 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426476 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426478 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:08:06.426485 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426481 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426484 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426486 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426489 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426492 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426494 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426497 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426500 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426503 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426505 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426508 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426511 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426513 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426516 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426518 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426521 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426523 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426526 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426529 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426531 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:08:06.426965 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426534 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426536 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426539 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426542 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426545 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426547 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426550 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426552 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426555 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426557 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426560 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426562 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426565 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426568 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426573 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426575 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426578 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426582 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426585 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:08:06.427513 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426588 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426591 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426594 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426596 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426599 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426601 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426604 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426607 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426610 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426612 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426616 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426620 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426622 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426626 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426628 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426631 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426634 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426637 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426641 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:08:06.428000 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426644 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426647 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426652 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426655 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426658 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426661 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426664 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426667 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426670 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426673 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426676 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426679 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426681 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426683 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426686 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426689 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426691 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426694 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426696 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:08:06.428462 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.426699 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428415 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428426 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428431 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428435 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428438 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428441 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428444 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428447 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428449 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428452 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428455 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428458 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428461 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428464 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428466 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428469 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428472 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428475 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:08:06.428922 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428477 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428480 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428482 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428485 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428489 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428492 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428495 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428498 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428501 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428504 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428507 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428509 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428512 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428514 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428517 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428521 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428523 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428526 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428530 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:08:06.429388 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428534 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428536 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428539 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428542 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428544 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428548 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428550 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428553 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428556 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428559 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428561 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428564 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428566 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428569 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428571 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428585 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428588 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428590 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428593 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428596 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:08:06.429861 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428598 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428601 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428603 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428606 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428609 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428612 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428614 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428617 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428620 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428623 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428626 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428629 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428632 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428634 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428637 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428639 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428642 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428646 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428648 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428651 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:08:06.430383 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428654 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428657 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428659 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428662 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428664 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428667 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428669 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428672 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428675 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428677 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428679 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428682 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428685 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428694 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428697 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428700 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428703 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428705 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428708 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428711 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:08:06.430891 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428715 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.428717 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429102 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429107 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429110 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429114 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429116 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429119 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429122 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429124 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429127 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429130 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429133 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429136 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429138 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429141 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429143 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429146 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429148 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429151 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:08:06.431370 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429154 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429156 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429159 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429164 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429167 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429170 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429172 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429175 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429178 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429183 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429186 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429189 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429193 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429196 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429198 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429201 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429204 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429206 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429209 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:08:06.431878 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429211 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429214 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429216 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429219 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429221 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429224 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429227 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429229 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429232 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429235 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429237 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429239 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429242 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429244 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429247 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429249 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429253 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429256 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429258 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429261 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:08:06.432345 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429263 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429266 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429271 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429274 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429277 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429280 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429283 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429286 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429288 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429291 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429293 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429296 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429299 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429301 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429304 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429307 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429311 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429313 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429316 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:08:06.432855 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429319 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429322 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429325 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429327 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429330 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429333 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429335 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429337 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429340 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429344 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429346 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429349 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429351 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429354 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429357 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429359 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429361 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429364 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429367 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429370 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:08:06.433310 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429372 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429375 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.429377 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429470 2566 flags.go:64] FLAG: --address="0.0.0.0" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429504 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429510 2566 flags.go:64] FLAG: --anonymous-auth="true" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429514 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429519 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429522 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429527 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429531 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429534 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429537 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429542 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429545 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429549 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429552 2566 flags.go:64] FLAG: --cgroup-root="" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429555 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429558 2566 flags.go:64] FLAG: --client-ca-file="" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429561 2566 flags.go:64] FLAG: --cloud-provider="external" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429564 2566 flags.go:64] FLAG: --cluster-dns="[]" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429569 2566 flags.go:64] FLAG: --cluster-domain="" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429572 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 17 07:08:06.433784 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429575 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429578 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429582 2566 flags.go:64] FLAG: --container-log-max-files="5" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429586 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429589 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429592 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429595 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429600 2566 flags.go:64] FLAG: --contention-profiling="false" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429603 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429606 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429609 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429612 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429616 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429619 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429622 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429624 2566 flags.go:64] FLAG: --enable-load-reader="false" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429627 2566 flags.go:64] FLAG: --enable-server="true" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429630 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429634 2566 flags.go:64] FLAG: --event-burst="100" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429637 2566 flags.go:64] FLAG: --event-qps="50" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429640 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429643 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429645 2566 flags.go:64] FLAG: --eviction-hard="" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429650 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 17 07:08:06.434336 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429653 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429656 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429659 2566 flags.go:64] FLAG: --eviction-soft="" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429662 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429665 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429668 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429671 2566 flags.go:64] FLAG: --experimental-mounter-path="" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429674 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429677 2566 flags.go:64] FLAG: --fail-swap-on="true" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429680 2566 flags.go:64] FLAG: --feature-gates="" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429684 2566 flags.go:64] FLAG: --file-check-frequency="20s" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429687 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429690 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429693 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429696 2566 flags.go:64] FLAG: --healthz-port="10248" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429700 2566 flags.go:64] FLAG: --help="false" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429703 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-142-90.ec2.internal" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429706 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429709 2566 flags.go:64] FLAG: --http-check-frequency="20s" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429712 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429715 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429719 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429721 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 17 07:08:06.434991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429724 2566 flags.go:64] FLAG: --image-service-endpoint="" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429727 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429730 2566 flags.go:64] FLAG: --kube-api-burst="100" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429733 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429736 2566 flags.go:64] FLAG: --kube-api-qps="50" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429739 2566 flags.go:64] FLAG: --kube-reserved="" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429742 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429745 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429748 2566 flags.go:64] FLAG: --kubelet-cgroups="" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429752 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429754 2566 flags.go:64] FLAG: --lock-file="" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429757 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429760 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429763 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429769 2566 flags.go:64] FLAG: --log-json-split-stream="false" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429771 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429774 2566 flags.go:64] FLAG: --log-text-split-stream="false" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429777 2566 flags.go:64] FLAG: --logging-format="text" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429780 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429783 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429786 2566 flags.go:64] FLAG: --manifest-url="" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429789 2566 flags.go:64] FLAG: --manifest-url-header="" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429793 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429795 2566 flags.go:64] FLAG: --max-open-files="1000000" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429799 2566 flags.go:64] FLAG: --max-pods="110" Sep 17 07:08:06.435637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429802 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429805 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429808 2566 flags.go:64] FLAG: --memory-manager-policy="None" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429811 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429814 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429816 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429819 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429826 2566 flags.go:64] FLAG: --node-status-max-images="50" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429830 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429833 2566 flags.go:64] FLAG: --oom-score-adj="-999" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429836 2566 flags.go:64] FLAG: --pod-cidr="" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429839 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429845 2566 flags.go:64] FLAG: --pod-manifest-path="" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429848 2566 flags.go:64] FLAG: --pod-max-pids="-1" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429851 2566 flags.go:64] FLAG: --pods-per-core="0" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429853 2566 flags.go:64] FLAG: --port="10250" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429856 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429867 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-026463d0d6d634c6d" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429871 2566 flags.go:64] FLAG: --qos-reserved="" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429874 2566 flags.go:64] FLAG: --read-only-port="10255" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429877 2566 flags.go:64] FLAG: --register-node="true" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429880 2566 flags.go:64] FLAG: --register-with-taints="" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429888 2566 flags.go:64] FLAG: --registry-burst="10" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429891 2566 flags.go:64] FLAG: --registry-qps="5" Sep 17 07:08:06.436254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429894 2566 flags.go:64] FLAG: --reserved-cpus="" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429897 2566 flags.go:64] FLAG: --reserved-memory="" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429901 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429904 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429907 2566 flags.go:64] FLAG: --rotate-certificates="false" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429910 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429913 2566 flags.go:64] FLAG: --runonce="false" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429916 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429918 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429921 2566 flags.go:64] FLAG: --seccomp-default="false" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429924 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429927 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429930 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429933 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429936 2566 flags.go:64] FLAG: --storage-driver-password="root" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429938 2566 flags.go:64] FLAG: --storage-driver-secure="false" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429941 2566 flags.go:64] FLAG: --storage-driver-table="stats" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429944 2566 flags.go:64] FLAG: --storage-driver-user="root" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429947 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429950 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429952 2566 flags.go:64] FLAG: --system-cgroups="" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429956 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429961 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429964 2566 flags.go:64] FLAG: --tls-cert-file="" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429967 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 17 07:08:06.436848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429970 2566 flags.go:64] FLAG: --tls-min-version="" Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429974 2566 flags.go:64] FLAG: --tls-private-key-file="" Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429977 2566 flags.go:64] FLAG: --topology-manager-policy="none" Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429980 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429982 2566 flags.go:64] FLAG: --topology-manager-scope="container" Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429987 2566 flags.go:64] FLAG: --v="2" Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429991 2566 flags.go:64] FLAG: --version="false" Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429995 2566 flags.go:64] FLAG: --vmodule="" Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.429999 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.430002 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430117 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430121 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430124 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430127 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430129 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430133 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430138 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430140 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430144 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430146 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430149 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430152 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:08:06.437587 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430155 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430158 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430162 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430165 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430169 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430172 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430175 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430178 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430180 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430183 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430185 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430188 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430191 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430194 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430196 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430200 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430203 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430205 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430208 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430211 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:08:06.438425 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430214 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430217 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430219 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430222 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430225 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430227 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430230 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430232 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430235 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430237 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430240 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430243 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430245 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430248 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430252 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430254 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430257 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430259 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430262 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:08:06.438920 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430264 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430267 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430269 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430272 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430275 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430277 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430280 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430284 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430288 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430290 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430293 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430296 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430299 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430303 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430306 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430310 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430313 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430315 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430319 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:08:06.439433 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430321 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430324 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430326 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430329 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430331 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430334 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430336 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430339 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430342 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430345 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430348 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430350 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430352 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430355 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430358 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430361 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430364 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430366 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430369 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430372 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:08:06.439898 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430375 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:08:06.440413 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430379 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:08:06.440413 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430382 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:08:06.440413 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430384 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:08:06.440413 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430387 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:08:06.440413 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430403 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:08:06.440413 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430406 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:08:06.440413 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430409 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:08:06.440413 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.430412 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:08:06.440413 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.430417 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 07:08:06.440413 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.439924 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 17 07:08:06.440413 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.440051 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 17 07:08:06.440413 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.440073 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 17 07:08:06.440413 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.440079 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 17 07:08:06.440413 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440137 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:08:06.440413 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440142 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440146 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440148 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440151 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440154 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440157 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440160 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440162 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440165 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440168 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440170 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440173 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440175 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440178 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440180 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440183 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440185 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440190 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440194 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:08:06.440797 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440198 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440201 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440204 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440208 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440210 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440214 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440217 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440221 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440225 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440229 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440233 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440235 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440239 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440241 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440244 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440247 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440249 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440252 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440255 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:08:06.441261 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440257 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440260 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440263 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440266 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440269 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440271 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440274 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440277 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440280 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440283 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440285 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440288 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440290 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440293 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440295 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440298 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440300 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440303 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440305 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440308 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:08:06.441783 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440310 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440313 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440316 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440319 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440322 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440325 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440327 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440330 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440332 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440335 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440337 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440340 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440343 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440346 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440349 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440352 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440354 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440357 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440360 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440362 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:08:06.442266 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440366 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440368 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440371 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440373 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440376 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440378 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440381 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440384 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440386 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440389 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440408 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440411 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440413 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440416 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440419 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440423 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440426 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440429 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440431 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:08:06.442768 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440434 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:08:06.443262 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.440440 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 07:08:06.443262 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440558 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:08:06.443262 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440563 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:08:06.443262 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440566 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:08:06.443262 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440569 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:08:06.443262 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440572 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:08:06.443262 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440575 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:08:06.443262 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440584 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:08:06.443262 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440588 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:08:06.443262 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440590 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:08:06.443262 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440593 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:08:06.443262 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440597 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:08:06.443262 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440602 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:08:06.443262 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440606 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440609 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440612 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440615 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440618 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440621 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440624 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440626 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440629 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440632 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440634 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440637 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440641 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440644 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440647 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440650 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440653 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440656 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440659 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:08:06.443637 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440662 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440664 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440667 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440670 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440673 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440675 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440678 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440681 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440684 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440687 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440689 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440692 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440694 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440697 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440701 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440704 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440707 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440709 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440712 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440715 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:08:06.444104 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440717 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440720 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440722 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440725 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440728 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440730 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440733 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440736 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440739 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440741 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440744 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440746 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440749 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440751 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440754 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440757 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440759 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440762 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440764 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440767 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:08:06.444617 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440769 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440772 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440774 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440777 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440779 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440782 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440784 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440787 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440789 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440792 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440795 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440797 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440800 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440803 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440805 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440808 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440811 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440813 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440816 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440819 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:08:06.445085 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440821 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:08:06.445715 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440824 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:08:06.445715 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440826 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:08:06.445715 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440829 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:08:06.445715 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440831 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:08:06.445715 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440834 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:08:06.445715 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440837 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:08:06.445715 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:06.440840 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:08:06.445715 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.440845 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 07:08:06.445715 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.440961 2566 server.go:962] "Client rotation is on, will bootstrap in background" Sep 17 07:08:06.445715 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.443376 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 17 07:08:06.445715 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.444465 2566 server.go:1019] "Starting client certificate rotation" Sep 17 07:08:06.445715 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.444561 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 07:08:06.445715 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.444611 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 07:08:06.447307 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.447292 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 07:08:06.448437 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.448420 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 07:08:06.464301 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.464275 2566 log.go:25] "Validated CRI v1 runtime API" Sep 17 07:08:06.467821 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.467805 2566 log.go:25] "Validated CRI v1 image API" Sep 17 07:08:06.468807 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.468788 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 17 07:08:06.473104 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.473082 2566 fs.go:135] Filesystem UUIDs: map[216609a9-82d0-43bb-9bd1-bbaad70eb8ab:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b5ce3ce9-d338-4321-9d05-dc9ff7d5948f:/dev/nvme0n1p4] Sep 17 07:08:06.473195 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.473103 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 17 07:08:06.474540 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.474519 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 07:08:06.479909 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.479788 2566 manager.go:216] Machine: {Timestamp:2026-09-17 07:08:06.478328544 +0000 UTC m=+0.375794185 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099674 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29c395619dd55b8fdf748974fa59a6 SystemUUID:ec29c395-619d-d55b-8fdf-748974fa59a6 BootID:21b42b86-a016-4d90-9f9b-2ed6093aaeca Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e7:c1:cd:5d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e7:c1:cd:5d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:26:d6:be:eb:0b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 17 07:08:06.479909 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.479896 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 17 07:08:06.480045 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.479983 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 17 07:08:06.483524 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.483491 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 17 07:08:06.483660 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.483528 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-90.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 17 07:08:06.484369 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.484360 2566 topology_manager.go:138] "Creating topology manager with none policy" Sep 17 07:08:06.484412 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.484372 2566 container_manager_linux.go:309] "Creating device plugin manager" Sep 17 07:08:06.484412 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.484380 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 07:08:06.485231 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.485220 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 07:08:06.486553 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.486537 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 17 07:08:06.491310 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.491296 2566 state_mem.go:36] "Initialized new in-memory state store" Sep 17 07:08:06.491443 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.491433 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 17 07:08:06.491485 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.491466 2566 kubelet.go:486] "Attempting to sync node with API server" Sep 17 07:08:06.491485 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.491476 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 17 07:08:06.491555 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.491492 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 17 07:08:06.491555 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.491502 2566 kubelet.go:398] "Adding apiserver pod source" Sep 17 07:08:06.491555 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.491511 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 17 07:08:06.491869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.491856 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 07:08:06.491914 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.491879 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 07:08:06.494218 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.494189 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 17 07:08:06.497312 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.497298 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 17 07:08:06.497383 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.497328 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 17 07:08:06.499149 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.499126 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-djnlm" Sep 17 07:08:06.499426 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.499412 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 17 07:08:06.499463 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.499433 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 17 07:08:06.499463 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.499440 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 17 07:08:06.499463 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.499445 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 17 07:08:06.499463 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.499455 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 17 07:08:06.499463 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.499460 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 17 07:08:06.499608 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.499473 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 17 07:08:06.499608 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.499479 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 17 07:08:06.499608 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.499485 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 17 07:08:06.499608 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.499490 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 17 07:08:06.499608 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.499504 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 17 07:08:06.499608 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.499511 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 17 07:08:06.500433 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.500418 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 17 07:08:06.500433 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.500432 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 17 07:08:06.500686 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.500666 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 17 07:08:06.500767 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.500748 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-90.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 17 07:08:06.502833 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.502816 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-90.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 17 07:08:06.504201 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.504189 2566 server.go:1268] "Started kubelet" Sep 17 07:08:06.504329 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.504262 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 17 07:08:06.504366 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.504352 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 17 07:08:06.504999 ip-10-0-142-90 systemd[1]: Started Kubernetes Kubelet. Sep 17 07:08:06.506341 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.505020 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 17 07:08:06.506341 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.505860 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 17 07:08:06.507756 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.507742 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-djnlm" Sep 17 07:08:06.507854 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.507840 2566 server.go:310] "Adding debug handlers to kubelet server" Sep 17 07:08:06.510097 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.510015 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 17 07:08:06.511279 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.510131 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-90.ec2.internal.18d609fd8bdbbad6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-90.ec2.internal,UID:ip-10-0-142-90.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-90.ec2.internal,},FirstTimestamp:2026-09-17 07:08:06.504159958 +0000 UTC m=+0.401625599,LastTimestamp:2026-09-17 07:08:06.504159958 +0000 UTC m=+0.401625599,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-90.ec2.internal,}" Sep 17 07:08:06.515247 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.515230 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 17 07:08:06.515322 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.515250 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 17 07:08:06.515322 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.515310 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-90.ec2.internal\" not found" Sep 17 07:08:06.516042 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.516026 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 17 07:08:06.516124 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.516027 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 17 07:08:06.516124 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.516072 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 17 07:08:06.516205 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.516166 2566 reconstruct.go:97] "Volume reconstruction finished" Sep 17 07:08:06.516205 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.516174 2566 reconciler.go:29] "Reconciler: start to sync state" Sep 17 07:08:06.516387 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.516374 2566 factory.go:153] Registering CRI-O factory Sep 17 07:08:06.516445 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.516429 2566 factory.go:223] Registration of the crio container factory successfully Sep 17 07:08:06.516514 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.516503 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 17 07:08:06.516553 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.516516 2566 factory.go:55] Registering systemd factory Sep 17 07:08:06.516553 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.516525 2566 factory.go:223] Registration of the systemd container factory successfully Sep 17 07:08:06.516553 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.516542 2566 factory.go:103] Registering Raw factory Sep 17 07:08:06.516658 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.516555 2566 manager.go:1196] Started watching for new ooms in manager Sep 17 07:08:06.516877 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.516865 2566 manager.go:349] Starting recovery of all containers Sep 17 07:08:06.517661 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.517638 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 17 07:08:06.517749 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.517642 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-90.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 17 07:08:06.528803 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.528783 2566 manager.go:354] Recovery completed Sep 17 07:08:06.533462 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.533448 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:08:06.535509 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.535484 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:08:06.535579 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.535517 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:08:06.535579 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.535529 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:08:06.536141 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.536127 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 17 07:08:06.536141 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.536135 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:08:06.536141 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.536139 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 17 07:08:06.536284 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.536155 2566 state_mem.go:36] "Initialized new in-memory state store" Sep 17 07:08:06.536830 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.536815 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:08:06.536883 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.536841 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:08:06.536883 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.536852 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:08:06.538232 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.538221 2566 policy_none.go:49] "None policy: Start" Sep 17 07:08:06.538282 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.538237 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 17 07:08:06.538282 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.538247 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 17 07:08:06.540365 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.540348 2566 policy_none.go:47] "Start" Sep 17 07:08:06.590736 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.588019 2566 manager.go:337] "Starting Device Plugin manager" Sep 17 07:08:06.590736 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.588059 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 17 07:08:06.590736 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.588072 2566 server.go:87] "Starting device plugin registration server" Sep 17 07:08:06.590736 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.588361 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 17 07:08:06.590736 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.588373 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 17 07:08:06.590736 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.588470 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 17 07:08:06.590736 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.588603 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 17 07:08:06.590736 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.588618 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 17 07:08:06.591109 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.590971 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 17 07:08:06.591109 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.591021 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-90.ec2.internal\" not found" Sep 17 07:08:06.610581 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.610536 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 17 07:08:06.611826 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.611810 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 17 07:08:06.611903 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.611871 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 17 07:08:06.611903 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.611897 2566 kubelet.go:2448] "Starting kubelet main sync loop" Sep 17 07:08:06.612016 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.611989 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 17 07:08:06.614529 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.614507 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:08:06.689246 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.689162 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:08:06.690111 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.690092 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:08:06.690225 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.690122 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:08:06.690225 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.690134 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:08:06.690225 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.690159 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-90.ec2.internal" Sep 17 07:08:06.701556 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.701536 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-90.ec2.internal" Sep 17 07:08:06.701659 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.701561 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-90.ec2.internal\": node \"ip-10-0-142-90.ec2.internal\" not found" Sep 17 07:08:06.712610 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.712586 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-90.ec2.internal"] Sep 17 07:08:06.712696 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.712672 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:08:06.713998 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.713983 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:08:06.714096 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.714010 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:08:06.714096 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.714025 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:08:06.716312 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.716298 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:08:06.716488 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.716460 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal" Sep 17 07:08:06.716536 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.716495 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:08:06.716939 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.716915 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3dcb0e14c4d6a09db4500d0c827116f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal\" (UID: \"f3dcb0e14c4d6a09db4500d0c827116f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal" Sep 17 07:08:06.717034 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.716950 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b5d2ed14de66e6a088f1d600ae6da75a-config\") pod \"kube-apiserver-proxy-ip-10-0-142-90.ec2.internal\" (UID: \"b5d2ed14de66e6a088f1d600ae6da75a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-90.ec2.internal" Sep 17 07:08:06.717034 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.716976 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f3dcb0e14c4d6a09db4500d0c827116f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal\" (UID: \"f3dcb0e14c4d6a09db4500d0c827116f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal" Sep 17 07:08:06.717134 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.717069 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:08:06.717134 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.717080 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:08:06.717134 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.717097 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:08:06.717134 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.717104 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:08:06.717134 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.717110 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:08:06.717134 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.717119 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:08:06.719344 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.719327 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-90.ec2.internal" Sep 17 07:08:06.719344 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.719348 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:08:06.720001 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.719986 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:08:06.720062 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.720011 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:08:06.720062 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.720020 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:08:06.732017 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.731998 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-90.ec2.internal\" not found" Sep 17 07:08:06.745312 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.745287 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-90.ec2.internal\" not found" node="ip-10-0-142-90.ec2.internal" Sep 17 07:08:06.749825 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.749807 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-90.ec2.internal\" not found" node="ip-10-0-142-90.ec2.internal" Sep 17 07:08:06.817745 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.817713 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b5d2ed14de66e6a088f1d600ae6da75a-config\") pod \"kube-apiserver-proxy-ip-10-0-142-90.ec2.internal\" (UID: \"b5d2ed14de66e6a088f1d600ae6da75a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-90.ec2.internal" Sep 17 07:08:06.817903 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.817752 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f3dcb0e14c4d6a09db4500d0c827116f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal\" (UID: \"f3dcb0e14c4d6a09db4500d0c827116f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal" Sep 17 07:08:06.817903 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.817772 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3dcb0e14c4d6a09db4500d0c827116f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal\" (UID: \"f3dcb0e14c4d6a09db4500d0c827116f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal" Sep 17 07:08:06.817903 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.817829 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3dcb0e14c4d6a09db4500d0c827116f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal\" (UID: \"f3dcb0e14c4d6a09db4500d0c827116f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal" Sep 17 07:08:06.817903 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.817829 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f3dcb0e14c4d6a09db4500d0c827116f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal\" (UID: \"f3dcb0e14c4d6a09db4500d0c827116f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal" Sep 17 07:08:06.817903 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:06.817830 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b5d2ed14de66e6a088f1d600ae6da75a-config\") pod \"kube-apiserver-proxy-ip-10-0-142-90.ec2.internal\" (UID: \"b5d2ed14de66e6a088f1d600ae6da75a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-90.ec2.internal" Sep 17 07:08:06.832865 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.832837 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-90.ec2.internal\" not found" Sep 17 07:08:06.933465 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:06.933430 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-90.ec2.internal\" not found" Sep 17 07:08:07.034245 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:07.034170 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-90.ec2.internal\" not found" Sep 17 07:08:07.048339 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:07.048316 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal" Sep 17 07:08:07.053511 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:07.053143 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-90.ec2.internal" Sep 17 07:08:08.540635 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:08.540589 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-90.ec2.internal\" not found" Sep 17 07:08:08.641096 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:08.641058 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-90.ec2.internal\" not found" Sep 17 07:08:08.741671 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:08.741587 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-90.ec2.internal\" not found" Sep 17 07:08:08.769293 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.769271 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:08:08.837958 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.837931 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:08:08.849663 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.849504 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 17 07:08:08.849800 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.849764 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 17 07:08:08.849848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.849806 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 17 07:08:08.849848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.849806 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 17 07:08:08.897861 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.897829 2566 apiserver.go:52] "Watching apiserver" Sep 17 07:08:08.916872 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.916843 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 17 07:08:08.917202 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.917179 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-zslxf","openshift-image-registry/node-ca-kvftr","openshift-ovn-kubernetes/ovnkube-node-64lxj","kube-system/global-pull-secret-syncer-7jt2z","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7","openshift-dns/node-resolver-wffc2","openshift-multus/multus-additional-cni-plugins-dzr67","openshift-multus/multus-nx9rp","openshift-multus/network-metrics-daemon-z2l2s","openshift-network-diagnostics/network-check-target-vddmm","openshift-network-operator/iptables-alerter-8cl5p","kube-system/konnectivity-agent-9mr6f"] Sep 17 07:08:08.919686 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.919662 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:08.921101 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.921082 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal" Sep 17 07:08:08.921183 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.921107 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 17 07:08:08.921183 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.921115 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 07:03:07 +0000 UTC" deadline="2028-04-25 14:12:38.715687189 +0000 UTC" Sep 17 07:08:08.921249 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.921189 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14071h4m29.794501677s" Sep 17 07:08:08.921808 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.921793 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kvftr" Sep 17 07:08:08.922122 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.922099 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 17 07:08:08.922251 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.922237 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 17 07:08:08.922678 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.922660 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pz7rf\"" Sep 17 07:08:08.923997 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.923980 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:08.926268 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.926249 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:08.926344 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:08.926321 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:08.926949 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.926931 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-systemd\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:08.926997 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.926962 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-host\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:08.927030 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.927016 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-modprobe-d\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:08.927061 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.927039 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-sysctl-d\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:08.927094 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.927056 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-lib-modules\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:08.927094 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.927080 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttrk7\" (UniqueName: \"kubernetes.io/projected/b815c071-1510-4f92-a8be-9d9a49cba65a-kube-api-access-ttrk7\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:08.927152 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.927102 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsqbb\" (UniqueName: \"kubernetes.io/projected/7a9610b5-26e5-44c9-a253-ea18eaec24af-kube-api-access-wsqbb\") pod \"node-ca-kvftr\" (UID: \"7a9610b5-26e5-44c9-a253-ea18eaec24af\") " pod="openshift-image-registry/node-ca-kvftr" Sep 17 07:08:08.927152 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.927118 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-kubernetes\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:08.927152 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.927132 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-run\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:08.927247 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.927149 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b815c071-1510-4f92-a8be-9d9a49cba65a-tmp\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:08.927247 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.927171 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a9610b5-26e5-44c9-a253-ea18eaec24af-host\") pod \"node-ca-kvftr\" (UID: \"7a9610b5-26e5-44c9-a253-ea18eaec24af\") " pod="openshift-image-registry/node-ca-kvftr" Sep 17 07:08:08.927247 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.927206 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7a9610b5-26e5-44c9-a253-ea18eaec24af-serviceca\") pod \"node-ca-kvftr\" (UID: \"7a9610b5-26e5-44c9-a253-ea18eaec24af\") " pod="openshift-image-registry/node-ca-kvftr" Sep 17 07:08:08.927247 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.927223 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-sysconfig\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:08.927247 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.927242 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-sysctl-conf\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:08.927419 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.927260 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-sys\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:08.927419 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.927280 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-var-lib-kubelet\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:08.927419 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.927300 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-tuned\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:08.928372 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.928354 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:08.929637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.929617 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 17 07:08:08.929726 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.929622 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 17 07:08:08.929726 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.929622 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 17 07:08:08.929726 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.929690 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 17 07:08:08.930029 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.930014 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5mnnn\"" Sep 17 07:08:08.930029 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.930021 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 17 07:08:08.930591 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.930575 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wffc2" Sep 17 07:08:08.931256 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.931237 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 17 07:08:08.931322 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.931269 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4gzfq\"" Sep 17 07:08:08.931402 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.931382 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 17 07:08:08.931513 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.931497 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x82mg\"" Sep 17 07:08:08.931574 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.931511 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 17 07:08:08.931574 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.931499 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 17 07:08:08.931652 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.931505 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 17 07:08:08.931652 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.931593 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 17 07:08:08.932325 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.932310 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 17 07:08:08.933027 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.932996 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:08.933979 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.933910 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 17 07:08:08.933979 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.933931 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 17 07:08:08.933979 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.933937 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7zx9n\"" Sep 17 07:08:08.934900 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.934870 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kgxzz\"" Sep 17 07:08:08.934984 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.934956 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 17 07:08:08.935070 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.935053 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 17 07:08:08.935418 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.935401 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 17 07:08:08.935600 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.935586 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nx9rp" Sep 17 07:08:08.935923 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.935904 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 17 07:08:08.936004 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.935992 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 17 07:08:08.938152 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.938132 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:08.938239 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:08.938198 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:08.938432 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.938416 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 17 07:08:08.938680 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.938666 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qzn49\"" Sep 17 07:08:08.940740 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.940721 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:08.940837 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:08.940819 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:08.941670 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.941653 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 07:08:08.943351 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.943334 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8cl5p" Sep 17 07:08:08.945353 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.945326 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 07:08:08.945559 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.945420 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-90.ec2.internal" Sep 17 07:08:08.945630 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.945583 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal"] Sep 17 07:08:08.945695 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.945680 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9mr6f" Sep 17 07:08:08.946188 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.946167 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 17 07:08:08.946322 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.946305 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 17 07:08:08.946514 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.946498 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 17 07:08:08.946632 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.946612 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c4r2m\"" Sep 17 07:08:08.948853 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.948835 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-28jkm\"" Sep 17 07:08:08.948971 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.948871 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 17 07:08:08.948971 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.948925 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 17 07:08:08.955732 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.955712 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-90.ec2.internal"] Sep 17 07:08:08.955850 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.955759 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 07:08:08.968940 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.968917 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2tzrp" Sep 17 07:08:08.985436 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:08.985411 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2tzrp" Sep 17 07:08:09.023072 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.023048 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 17 07:08:09.027588 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027569 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-lib-modules\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.027684 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027595 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttrk7\" (UniqueName: \"kubernetes.io/projected/b815c071-1510-4f92-a8be-9d9a49cba65a-kube-api-access-ttrk7\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.027684 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027615 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-run-ovn-kubernetes\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.027684 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027637 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3953be2c-e8eb-4172-88fa-8d04babc59fd-ovn-node-metrics-cert\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.027684 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027652 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-etc-selinux\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.027887 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027690 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8afb189f-1edd-473f-92dc-f923a60d691e-cnibin\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.027887 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027731 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8afb189f-1edd-473f-92dc-f923a60d691e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.027887 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027748 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-lib-modules\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.027887 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027758 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-system-cni-dir\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.027887 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027797 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-sys-fs\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.027887 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027827 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a9610b5-26e5-44c9-a253-ea18eaec24af-host\") pod \"node-ca-kvftr\" (UID: \"7a9610b5-26e5-44c9-a253-ea18eaec24af\") " pod="openshift-image-registry/node-ca-kvftr" Sep 17 07:08:09.027887 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027877 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a9610b5-26e5-44c9-a253-ea18eaec24af-host\") pod \"node-ca-kvftr\" (UID: \"7a9610b5-26e5-44c9-a253-ea18eaec24af\") " pod="openshift-image-registry/node-ca-kvftr" Sep 17 07:08:09.028223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027885 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e59c89a5-bae3-48e6-bdb5-f52094590f45-dbus\") pod \"global-pull-secret-syncer-7jt2z\" (UID: \"e59c89a5-bae3-48e6-bdb5-f52094590f45\") " pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:09.028223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027912 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/abd2409d-642b-4c1e-a55c-a4a2c702b553-iptables-alerter-script\") pod \"iptables-alerter-8cl5p\" (UID: \"abd2409d-642b-4c1e-a55c-a4a2c702b553\") " pod="openshift-network-operator/iptables-alerter-8cl5p" Sep 17 07:08:09.028223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027938 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-run-ovn\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.028223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027972 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-log-socket\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.028223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.027995 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8afb189f-1edd-473f-92dc-f923a60d691e-system-cni-dir\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.028223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028013 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-kubelet\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.028223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028027 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-node-log\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.028223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028041 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-device-dir\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.028223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028072 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e94051ac-fe49-4338-ab5f-69979421a9cb-hosts-file\") pod \"node-resolver-wffc2\" (UID: \"e94051ac-fe49-4338-ab5f-69979421a9cb\") " pod="openshift-dns/node-resolver-wffc2" Sep 17 07:08:09.028223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028105 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8afb189f-1edd-473f-92dc-f923a60d691e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.028223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028139 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.028223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028162 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-hostroot\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.028223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028184 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/93339a00-d0f0-4f0d-9449-bbb4566c2b93-multus-daemon-config\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.028223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028208 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs\") pod \"network-metrics-daemon-z2l2s\" (UID: \"dc4c353d-0b67-4b66-a052-dd8397466d6f\") " pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:09.028223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028229 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh2hb\" (UniqueName: \"kubernetes.io/projected/abd2409d-642b-4c1e-a55c-a4a2c702b553-kube-api-access-kh2hb\") pod \"iptables-alerter-8cl5p\" (UID: \"abd2409d-642b-4c1e-a55c-a4a2c702b553\") " pod="openshift-network-operator/iptables-alerter-8cl5p" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028255 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-sysctl-d\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028279 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-run-netns\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028303 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-var-lib-openvswitch\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028328 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8k4j\" (UniqueName: \"kubernetes.io/projected/3953be2c-e8eb-4172-88fa-8d04babc59fd-kube-api-access-p8k4j\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028354 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/93339a00-d0f0-4f0d-9449-bbb4566c2b93-cni-binary-copy\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028379 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-sysctl-d\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028390 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-var-lib-cni-multus\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028416 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-sysconfig\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028439 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-tuned\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028462 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e59c89a5-bae3-48e6-bdb5-f52094590f45-kubelet-config\") pod \"global-pull-secret-syncer-7jt2z\" (UID: \"e59c89a5-bae3-48e6-bdb5-f52094590f45\") " pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028486 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-socket-dir\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028512 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-registration-dir\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028535 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-multus-cni-dir\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028558 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8afb189f-1edd-473f-92dc-f923a60d691e-cni-binary-copy\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028573 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-sysconfig\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028580 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-cnibin\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.028929 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028618 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nkbl\" (UniqueName: \"kubernetes.io/projected/dc4c353d-0b67-4b66-a052-dd8397466d6f-kube-api-access-2nkbl\") pod \"network-metrics-daemon-z2l2s\" (UID: \"dc4c353d-0b67-4b66-a052-dd8397466d6f\") " pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028666 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsqbb\" (UniqueName: \"kubernetes.io/projected/7a9610b5-26e5-44c9-a253-ea18eaec24af-kube-api-access-wsqbb\") pod \"node-ca-kvftr\" (UID: \"7a9610b5-26e5-44c9-a253-ea18eaec24af\") " pod="openshift-image-registry/node-ca-kvftr" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028715 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg4sk\" (UniqueName: \"kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk\") pod \"network-check-target-vddmm\" (UID: \"3b975ace-eff0-4687-845a-c9607f77ac13\") " pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028744 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhvxw\" (UniqueName: \"kubernetes.io/projected/8afb189f-1edd-473f-92dc-f923a60d691e-kube-api-access-vhvxw\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028768 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-run\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028795 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028810 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b815c071-1510-4f92-a8be-9d9a49cba65a-tmp\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028836 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-etc-openvswitch\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028860 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8afb189f-1edd-473f-92dc-f923a60d691e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028906 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-sys\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028969 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-sys\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028970 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-run\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.028983 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-var-lib-kubelet\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029015 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abd2409d-642b-4c1e-a55c-a4a2c702b553-host-slash\") pod \"iptables-alerter-8cl5p\" (UID: \"abd2409d-642b-4c1e-a55c-a4a2c702b553\") " pod="openshift-network-operator/iptables-alerter-8cl5p" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029037 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a9bab8de-9d11-4e20-814c-74e2afb9eb19-agent-certs\") pod \"konnectivity-agent-9mr6f\" (UID: \"a9bab8de-9d11-4e20-814c-74e2afb9eb19\") " pod="kube-system/konnectivity-agent-9mr6f" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029036 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-var-lib-kubelet\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029058 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-cni-bin\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.029572 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029082 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-run-netns\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029103 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-etc-kubernetes\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029121 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm5zf\" (UniqueName: \"kubernetes.io/projected/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-kube-api-access-fm5zf\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029144 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-systemd\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029189 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-systemd\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029210 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-systemd-units\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029231 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-run-openvswitch\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029252 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3953be2c-e8eb-4172-88fa-8d04babc59fd-ovnkube-config\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029269 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3953be2c-e8eb-4172-88fa-8d04babc59fd-ovnkube-script-lib\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029284 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e94051ac-fe49-4338-ab5f-69979421a9cb-tmp-dir\") pod \"node-resolver-wffc2\" (UID: \"e94051ac-fe49-4338-ab5f-69979421a9cb\") " pod="openshift-dns/node-resolver-wffc2" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029302 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3953be2c-e8eb-4172-88fa-8d04babc59fd-env-overrides\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029316 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-host\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029331 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-modprobe-d\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029348 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret\") pod \"global-pull-secret-syncer-7jt2z\" (UID: \"e59c89a5-bae3-48e6-bdb5-f52094590f45\") " pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029366 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-cni-netd\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029401 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029408 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-host\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.030164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029425 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-multus-socket-dir-parent\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029442 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-run-k8s-cni-cncf-io\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029466 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-kubernetes\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029471 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-modprobe-d\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029490 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7a9610b5-26e5-44c9-a253-ea18eaec24af-serviceca\") pod \"node-ca-kvftr\" (UID: \"7a9610b5-26e5-44c9-a253-ea18eaec24af\") " pod="openshift-image-registry/node-ca-kvftr" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029506 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a9bab8de-9d11-4e20-814c-74e2afb9eb19-konnectivity-ca\") pod \"konnectivity-agent-9mr6f\" (UID: \"a9bab8de-9d11-4e20-814c-74e2afb9eb19\") " pod="kube-system/konnectivity-agent-9mr6f" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029527 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tzcg\" (UniqueName: \"kubernetes.io/projected/e94051ac-fe49-4338-ab5f-69979421a9cb-kube-api-access-4tzcg\") pod \"node-resolver-wffc2\" (UID: \"e94051ac-fe49-4338-ab5f-69979421a9cb\") " pod="openshift-dns/node-resolver-wffc2" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029534 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-kubernetes\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029548 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-var-lib-cni-bin\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029570 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-var-lib-kubelet\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029604 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-sysctl-conf\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029631 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-slash\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029654 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-multus-conf-dir\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029677 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-run-multus-certs\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029700 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q499\" (UniqueName: \"kubernetes.io/projected/93339a00-d0f0-4f0d-9449-bbb4566c2b93-kube-api-access-6q499\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029722 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-run-systemd\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029875 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-sysctl-conf\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.030682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.029954 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7a9610b5-26e5-44c9-a253-ea18eaec24af-serviceca\") pod \"node-ca-kvftr\" (UID: \"7a9610b5-26e5-44c9-a253-ea18eaec24af\") " pod="openshift-image-registry/node-ca-kvftr" Sep 17 07:08:09.031815 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.031778 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b815c071-1510-4f92-a8be-9d9a49cba65a-tmp\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.031903 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.031820 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b815c071-1510-4f92-a8be-9d9a49cba65a-etc-tuned\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.036275 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.036250 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttrk7\" (UniqueName: \"kubernetes.io/projected/b815c071-1510-4f92-a8be-9d9a49cba65a-kube-api-access-ttrk7\") pod \"tuned-zslxf\" (UID: \"b815c071-1510-4f92-a8be-9d9a49cba65a\") " pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.037157 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.037136 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsqbb\" (UniqueName: \"kubernetes.io/projected/7a9610b5-26e5-44c9-a253-ea18eaec24af-kube-api-access-wsqbb\") pod \"node-ca-kvftr\" (UID: \"7a9610b5-26e5-44c9-a253-ea18eaec24af\") " pod="openshift-image-registry/node-ca-kvftr" Sep 17 07:08:09.107316 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:09.107269 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3dcb0e14c4d6a09db4500d0c827116f.slice/crio-7a4e9b9275c526ba8bb8a2620dd561aeb5d62a5c056d051fc696e0da3cb5b7f5 WatchSource:0}: Error finding container 7a4e9b9275c526ba8bb8a2620dd561aeb5d62a5c056d051fc696e0da3cb5b7f5: Status 404 returned error can't find the container with id 7a4e9b9275c526ba8bb8a2620dd561aeb5d62a5c056d051fc696e0da3cb5b7f5 Sep 17 07:08:09.111383 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.111367 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130460 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e59c89a5-bae3-48e6-bdb5-f52094590f45-dbus\") pod \"global-pull-secret-syncer-7jt2z\" (UID: \"e59c89a5-bae3-48e6-bdb5-f52094590f45\") " pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130515 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/abd2409d-642b-4c1e-a55c-a4a2c702b553-iptables-alerter-script\") pod \"iptables-alerter-8cl5p\" (UID: \"abd2409d-642b-4c1e-a55c-a4a2c702b553\") " pod="openshift-network-operator/iptables-alerter-8cl5p" Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130548 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-run-ovn\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130572 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-log-socket\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130602 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8afb189f-1edd-473f-92dc-f923a60d691e-system-cni-dir\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130632 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-kubelet\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130661 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-node-log\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130684 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-device-dir\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130713 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e94051ac-fe49-4338-ab5f-69979421a9cb-hosts-file\") pod \"node-resolver-wffc2\" (UID: \"e94051ac-fe49-4338-ab5f-69979421a9cb\") " pod="openshift-dns/node-resolver-wffc2" Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130742 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8afb189f-1edd-473f-92dc-f923a60d691e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130772 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130819 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-hostroot\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130844 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/93339a00-d0f0-4f0d-9449-bbb4566c2b93-multus-daemon-config\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130873 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs\") pod \"network-metrics-daemon-z2l2s\" (UID: \"dc4c353d-0b67-4b66-a052-dd8397466d6f\") " pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130900 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh2hb\" (UniqueName: \"kubernetes.io/projected/abd2409d-642b-4c1e-a55c-a4a2c702b553-kube-api-access-kh2hb\") pod \"iptables-alerter-8cl5p\" (UID: \"abd2409d-642b-4c1e-a55c-a4a2c702b553\") " pod="openshift-network-operator/iptables-alerter-8cl5p" Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130930 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-run-netns\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.132313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130953 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-var-lib-openvswitch\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.130985 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8k4j\" (UniqueName: \"kubernetes.io/projected/3953be2c-e8eb-4172-88fa-8d04babc59fd-kube-api-access-p8k4j\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131013 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/93339a00-d0f0-4f0d-9449-bbb4566c2b93-cni-binary-copy\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131041 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-var-lib-cni-multus\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131073 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e59c89a5-bae3-48e6-bdb5-f52094590f45-kubelet-config\") pod \"global-pull-secret-syncer-7jt2z\" (UID: \"e59c89a5-bae3-48e6-bdb5-f52094590f45\") " pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131103 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-socket-dir\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131126 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-registration-dir\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131155 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-multus-cni-dir\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131183 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8afb189f-1edd-473f-92dc-f923a60d691e-cni-binary-copy\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131212 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-cnibin\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131242 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nkbl\" (UniqueName: \"kubernetes.io/projected/dc4c353d-0b67-4b66-a052-dd8397466d6f-kube-api-access-2nkbl\") pod \"network-metrics-daemon-z2l2s\" (UID: \"dc4c353d-0b67-4b66-a052-dd8397466d6f\") " pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131268 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg4sk\" (UniqueName: \"kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk\") pod \"network-check-target-vddmm\" (UID: \"3b975ace-eff0-4687-845a-c9607f77ac13\") " pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131307 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhvxw\" (UniqueName: \"kubernetes.io/projected/8afb189f-1edd-473f-92dc-f923a60d691e-kube-api-access-vhvxw\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131346 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-etc-openvswitch\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131375 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8afb189f-1edd-473f-92dc-f923a60d691e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131402 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abd2409d-642b-4c1e-a55c-a4a2c702b553-host-slash\") pod \"iptables-alerter-8cl5p\" (UID: \"abd2409d-642b-4c1e-a55c-a4a2c702b553\") " pod="openshift-network-operator/iptables-alerter-8cl5p" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131432 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a9bab8de-9d11-4e20-814c-74e2afb9eb19-agent-certs\") pod \"konnectivity-agent-9mr6f\" (UID: \"a9bab8de-9d11-4e20-814c-74e2afb9eb19\") " pod="kube-system/konnectivity-agent-9mr6f" Sep 17 07:08:09.133102 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131461 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-cni-bin\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131497 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-run-netns\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131525 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-etc-kubernetes\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131556 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm5zf\" (UniqueName: \"kubernetes.io/projected/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-kube-api-access-fm5zf\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131588 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-systemd-units\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131612 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-run-openvswitch\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131643 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3953be2c-e8eb-4172-88fa-8d04babc59fd-ovnkube-config\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131676 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3953be2c-e8eb-4172-88fa-8d04babc59fd-ovnkube-script-lib\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131706 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e94051ac-fe49-4338-ab5f-69979421a9cb-tmp-dir\") pod \"node-resolver-wffc2\" (UID: \"e94051ac-fe49-4338-ab5f-69979421a9cb\") " pod="openshift-dns/node-resolver-wffc2" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131730 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3953be2c-e8eb-4172-88fa-8d04babc59fd-env-overrides\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131763 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret\") pod \"global-pull-secret-syncer-7jt2z\" (UID: \"e59c89a5-bae3-48e6-bdb5-f52094590f45\") " pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131808 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-cni-netd\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131841 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131872 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-multus-socket-dir-parent\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131896 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-run-k8s-cni-cncf-io\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131927 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a9bab8de-9d11-4e20-814c-74e2afb9eb19-konnectivity-ca\") pod \"konnectivity-agent-9mr6f\" (UID: \"a9bab8de-9d11-4e20-814c-74e2afb9eb19\") " pod="kube-system/konnectivity-agent-9mr6f" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131958 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tzcg\" (UniqueName: \"kubernetes.io/projected/e94051ac-fe49-4338-ab5f-69979421a9cb-kube-api-access-4tzcg\") pod \"node-resolver-wffc2\" (UID: \"e94051ac-fe49-4338-ab5f-69979421a9cb\") " pod="openshift-dns/node-resolver-wffc2" Sep 17 07:08:09.133811 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.131988 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-var-lib-cni-bin\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132021 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-var-lib-kubelet\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132049 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-slash\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132077 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-multus-conf-dir\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132108 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-run-multus-certs\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132137 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q499\" (UniqueName: \"kubernetes.io/projected/93339a00-d0f0-4f0d-9449-bbb4566c2b93-kube-api-access-6q499\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132161 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-run-systemd\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132194 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-run-ovn-kubernetes\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132224 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3953be2c-e8eb-4172-88fa-8d04babc59fd-ovn-node-metrics-cert\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132253 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-etc-selinux\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132281 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8afb189f-1edd-473f-92dc-f923a60d691e-cnibin\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132413 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8afb189f-1edd-473f-92dc-f923a60d691e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132435 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-registration-dir\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132447 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-system-cni-dir\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132473 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-sys-fs\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132525 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-sys-fs\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132654 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-cnibin\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.134637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132694 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-system-cni-dir\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132749 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-multus-cni-dir\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132439 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-multus-conf-dir\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132939 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-device-dir\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132947 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.132963 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-cni-bin\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:09.133397 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.133416 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8afb189f-1edd-473f-92dc-f923a60d691e-cnibin\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.133467 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-slash\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:09.133480 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs podName:dc4c353d-0b67-4b66-a052-dd8397466d6f nodeName:}" failed. No retries permitted until 2026-09-17 07:08:09.633458297 +0000 UTC m=+2.125533080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs") pod "network-metrics-daemon-z2l2s" (UID: "dc4c353d-0b67-4b66-a052-dd8397466d6f") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.133397 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-run-netns\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.133503 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-socket-dir\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.133584 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e94051ac-fe49-4338-ab5f-69979421a9cb-tmp-dir\") pod \"node-resolver-wffc2\" (UID: \"e94051ac-fe49-4338-ab5f-69979421a9cb\") " pod="openshift-dns/node-resolver-wffc2" Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.133642 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8afb189f-1edd-473f-92dc-f923a60d691e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.133688 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-run-netns\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.133716 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-etc-openvswitch\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.133749 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abd2409d-642b-4c1e-a55c-a4a2c702b553-host-slash\") pod \"iptables-alerter-8cl5p\" (UID: \"abd2409d-642b-4c1e-a55c-a4a2c702b553\") " pod="openshift-network-operator/iptables-alerter-8cl5p" Sep 17 07:08:09.135335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.133772 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-var-lib-openvswitch\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.133896 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/93339a00-d0f0-4f0d-9449-bbb4566c2b93-multus-daemon-config\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.133896 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a9bab8de-9d11-4e20-814c-74e2afb9eb19-konnectivity-ca\") pod \"konnectivity-agent-9mr6f\" (UID: \"a9bab8de-9d11-4e20-814c-74e2afb9eb19\") " pod="kube-system/konnectivity-agent-9mr6f" Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.133940 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-run-openvswitch\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.133955 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-systemd-units\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.133980 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e59c89a5-bae3-48e6-bdb5-f52094590f45-dbus\") pod \"global-pull-secret-syncer-7jt2z\" (UID: \"e59c89a5-bae3-48e6-bdb5-f52094590f45\") " pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134000 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-cni-netd\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134018 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-run-ovn\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134043 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134075 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/abd2409d-642b-4c1e-a55c-a4a2c702b553-iptables-alerter-script\") pod \"iptables-alerter-8cl5p\" (UID: \"abd2409d-642b-4c1e-a55c-a4a2c702b553\") " pod="openshift-network-operator/iptables-alerter-8cl5p" Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134081 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e59c89a5-bae3-48e6-bdb5-f52094590f45-kubelet-config\") pod \"global-pull-secret-syncer-7jt2z\" (UID: \"e59c89a5-bae3-48e6-bdb5-f52094590f45\") " pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134142 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-var-lib-cni-multus\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:09.134228 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134228 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8afb189f-1edd-473f-92dc-f923a60d691e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134273 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-run-multus-certs\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:09.134282 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret podName:e59c89a5-bae3-48e6-bdb5-f52094590f45 nodeName:}" failed. No retries permitted until 2026-09-17 07:08:09.63427117 +0000 UTC m=+2.126345945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret") pod "global-pull-secret-syncer-7jt2z" (UID: "e59c89a5-bae3-48e6-bdb5-f52094590f45") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134311 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8afb189f-1edd-473f-92dc-f923a60d691e-system-cni-dir\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.135869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134352 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-kubelet\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134403 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-etc-selinux\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134440 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-log-socket\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134476 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3953be2c-e8eb-4172-88fa-8d04babc59fd-env-overrides\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134480 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-node-log\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134539 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e94051ac-fe49-4338-ab5f-69979421a9cb-hosts-file\") pod \"node-resolver-wffc2\" (UID: \"e94051ac-fe49-4338-ab5f-69979421a9cb\") " pod="openshift-dns/node-resolver-wffc2" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134536 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3953be2c-e8eb-4172-88fa-8d04babc59fd-ovnkube-config\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134598 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8afb189f-1edd-473f-92dc-f923a60d691e-cni-binary-copy\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134633 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-run-k8s-cni-cncf-io\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134682 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-hostroot\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134716 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-var-lib-cni-bin\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.134925 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-run-systemd\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.135049 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-etc-kubernetes\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.135097 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-host-var-lib-kubelet\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.135113 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3953be2c-e8eb-4172-88fa-8d04babc59fd-ovnkube-script-lib\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.135136 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/93339a00-d0f0-4f0d-9449-bbb4566c2b93-cni-binary-copy\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.135136 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3953be2c-e8eb-4172-88fa-8d04babc59fd-host-run-ovn-kubernetes\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.135164 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8afb189f-1edd-473f-92dc-f923a60d691e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.136335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.135166 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/93339a00-d0f0-4f0d-9449-bbb4566c2b93-multus-socket-dir-parent\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.136849 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.136799 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3953be2c-e8eb-4172-88fa-8d04babc59fd-ovn-node-metrics-cert\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.136849 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.136830 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a9bab8de-9d11-4e20-814c-74e2afb9eb19-agent-certs\") pod \"konnectivity-agent-9mr6f\" (UID: \"a9bab8de-9d11-4e20-814c-74e2afb9eb19\") " pod="kube-system/konnectivity-agent-9mr6f" Sep 17 07:08:09.147767 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.147745 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh2hb\" (UniqueName: \"kubernetes.io/projected/abd2409d-642b-4c1e-a55c-a4a2c702b553-kube-api-access-kh2hb\") pod \"iptables-alerter-8cl5p\" (UID: \"abd2409d-642b-4c1e-a55c-a4a2c702b553\") " pod="openshift-network-operator/iptables-alerter-8cl5p" Sep 17 07:08:09.149278 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:09.149261 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:08:09.149364 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:09.149284 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:08:09.149364 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:09.149294 2566 projected.go:196] Error preparing data for projected volume kube-api-access-hg4sk for pod openshift-network-diagnostics/network-check-target-vddmm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:08:09.149364 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:09.149336 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk podName:3b975ace-eff0-4687-845a-c9607f77ac13 nodeName:}" failed. No retries permitted until 2026-09-17 07:08:09.649324506 +0000 UTC m=+2.141399277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hg4sk" (UniqueName: "kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk") pod "network-check-target-vddmm" (UID: "3b975ace-eff0-4687-845a-c9607f77ac13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:08:09.152921 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.152900 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q499\" (UniqueName: \"kubernetes.io/projected/93339a00-d0f0-4f0d-9449-bbb4566c2b93-kube-api-access-6q499\") pod \"multus-nx9rp\" (UID: \"93339a00-d0f0-4f0d-9449-bbb4566c2b93\") " pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.153238 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.153212 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nkbl\" (UniqueName: \"kubernetes.io/projected/dc4c353d-0b67-4b66-a052-dd8397466d6f-kube-api-access-2nkbl\") pod \"network-metrics-daemon-z2l2s\" (UID: \"dc4c353d-0b67-4b66-a052-dd8397466d6f\") " pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:09.154118 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.154099 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8k4j\" (UniqueName: \"kubernetes.io/projected/3953be2c-e8eb-4172-88fa-8d04babc59fd-kube-api-access-p8k4j\") pod \"ovnkube-node-64lxj\" (UID: \"3953be2c-e8eb-4172-88fa-8d04babc59fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.154182 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.154161 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhvxw\" (UniqueName: \"kubernetes.io/projected/8afb189f-1edd-473f-92dc-f923a60d691e-kube-api-access-vhvxw\") pod \"multus-additional-cni-plugins-dzr67\" (UID: \"8afb189f-1edd-473f-92dc-f923a60d691e\") " pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.154303 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.154287 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm5zf\" (UniqueName: \"kubernetes.io/projected/9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f-kube-api-access-fm5zf\") pod \"aws-ebs-csi-driver-node-nl2t7\" (UID: \"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.157850 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.157833 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tzcg\" (UniqueName: \"kubernetes.io/projected/e94051ac-fe49-4338-ab5f-69979421a9cb-kube-api-access-4tzcg\") pod \"node-resolver-wffc2\" (UID: \"e94051ac-fe49-4338-ab5f-69979421a9cb\") " pod="openshift-dns/node-resolver-wffc2" Sep 17 07:08:09.204440 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:09.204409 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5d2ed14de66e6a088f1d600ae6da75a.slice/crio-9a380f29128b046944497109a3d76bf91b88b68fbe8efcc56ec3adbe138f4291 WatchSource:0}: Error finding container 9a380f29128b046944497109a3d76bf91b88b68fbe8efcc56ec3adbe138f4291: Status 404 returned error can't find the container with id 9a380f29128b046944497109a3d76bf91b88b68fbe8efcc56ec3adbe138f4291 Sep 17 07:08:09.239277 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.239242 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zslxf" Sep 17 07:08:09.247602 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:09.247576 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb815c071_1510_4f92_a8be_9d9a49cba65a.slice/crio-ff602c604818d0d119aa9049582c0e2c56b04654680b36cb4d85073c306bcc02 WatchSource:0}: Error finding container ff602c604818d0d119aa9049582c0e2c56b04654680b36cb4d85073c306bcc02: Status 404 returned error can't find the container with id ff602c604818d0d119aa9049582c0e2c56b04654680b36cb4d85073c306bcc02 Sep 17 07:08:09.253809 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.253779 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kvftr" Sep 17 07:08:09.261872 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:09.261852 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a9610b5_26e5_44c9_a253_ea18eaec24af.slice/crio-22c4e4b3ea05d975f67fe8be31147d1603e45786b44f584abfcdae23a0d605c6 WatchSource:0}: Error finding container 22c4e4b3ea05d975f67fe8be31147d1603e45786b44f584abfcdae23a0d605c6: Status 404 returned error can't find the container with id 22c4e4b3ea05d975f67fe8be31147d1603e45786b44f584abfcdae23a0d605c6 Sep 17 07:08:09.263041 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.263026 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:09.270600 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:09.270577 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3953be2c_e8eb_4172_88fa_8d04babc59fd.slice/crio-bc43acc550283099c132f7f310b5fa8088613edfdb1a61c78b2ad006ab59542e WatchSource:0}: Error finding container bc43acc550283099c132f7f310b5fa8088613edfdb1a61c78b2ad006ab59542e: Status 404 returned error can't find the container with id bc43acc550283099c132f7f310b5fa8088613edfdb1a61c78b2ad006ab59542e Sep 17 07:08:09.276771 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.276751 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" Sep 17 07:08:09.285360 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:09.285330 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aed1d6e_0d13_4262_a9ad_48fbcd4e8d5f.slice/crio-dc512086dd46bdfe49ac59be09af96dd21fb313114d25b857024dbbd45d00036 WatchSource:0}: Error finding container dc512086dd46bdfe49ac59be09af96dd21fb313114d25b857024dbbd45d00036: Status 404 returned error can't find the container with id dc512086dd46bdfe49ac59be09af96dd21fb313114d25b857024dbbd45d00036 Sep 17 07:08:09.290333 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.290316 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wffc2" Sep 17 07:08:09.297649 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:09.297629 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode94051ac_fe49_4338_ab5f_69979421a9cb.slice/crio-2a2e49cacc0b1eba84c5fafae4f4ba357874cd0acf35dea91652bf6b522b5393 WatchSource:0}: Error finding container 2a2e49cacc0b1eba84c5fafae4f4ba357874cd0acf35dea91652bf6b522b5393: Status 404 returned error can't find the container with id 2a2e49cacc0b1eba84c5fafae4f4ba357874cd0acf35dea91652bf6b522b5393 Sep 17 07:08:09.310460 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.310439 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dzr67" Sep 17 07:08:09.318215 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:09.318195 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8afb189f_1edd_473f_92dc_f923a60d691e.slice/crio-9a5c219b5b2c273b5d2e2602d996565d15f72778ebf7a4fd8c7a211fa99ef4e3 WatchSource:0}: Error finding container 9a5c219b5b2c273b5d2e2602d996565d15f72778ebf7a4fd8c7a211fa99ef4e3: Status 404 returned error can't find the container with id 9a5c219b5b2c273b5d2e2602d996565d15f72778ebf7a4fd8c7a211fa99ef4e3 Sep 17 07:08:09.323175 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.323158 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nx9rp" Sep 17 07:08:09.333006 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:09.332979 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93339a00_d0f0_4f0d_9449_bbb4566c2b93.slice/crio-85ff539ba63caf9d6b45df8e8b65692660fd492da77a87a14be139cf16a879de WatchSource:0}: Error finding container 85ff539ba63caf9d6b45df8e8b65692660fd492da77a87a14be139cf16a879de: Status 404 returned error can't find the container with id 85ff539ba63caf9d6b45df8e8b65692660fd492da77a87a14be139cf16a879de Sep 17 07:08:09.344470 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.344449 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8cl5p" Sep 17 07:08:09.349266 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.349251 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9mr6f" Sep 17 07:08:09.353673 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:09.353652 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabd2409d_642b_4c1e_a55c_a4a2c702b553.slice/crio-d02fd1f7cc243c5db74267911f66ed97b1522880f7c13528a38ccdab0c78fb25 WatchSource:0}: Error finding container d02fd1f7cc243c5db74267911f66ed97b1522880f7c13528a38ccdab0c78fb25: Status 404 returned error can't find the container with id d02fd1f7cc243c5db74267911f66ed97b1522880f7c13528a38ccdab0c78fb25 Sep 17 07:08:09.358532 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:09.358511 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9bab8de_9d11_4e20_814c_74e2afb9eb19.slice/crio-f3b3e614f1bf2915941900128f7f16496be0127e98963c0144ce9eed95690522 WatchSource:0}: Error finding container f3b3e614f1bf2915941900128f7f16496be0127e98963c0144ce9eed95690522: Status 404 returned error can't find the container with id f3b3e614f1bf2915941900128f7f16496be0127e98963c0144ce9eed95690522 Sep 17 07:08:09.464342 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.464320 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:08:09.636513 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.636474 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs\") pod \"network-metrics-daemon-z2l2s\" (UID: \"dc4c353d-0b67-4b66-a052-dd8397466d6f\") " pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:09.636674 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.636555 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret\") pod \"global-pull-secret-syncer-7jt2z\" (UID: \"e59c89a5-bae3-48e6-bdb5-f52094590f45\") " pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:09.636674 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:09.636593 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:08:09.636674 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:09.636669 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs podName:dc4c353d-0b67-4b66-a052-dd8397466d6f nodeName:}" failed. No retries permitted until 2026-09-17 07:08:10.636651941 +0000 UTC m=+3.128726734 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs") pod "network-metrics-daemon-z2l2s" (UID: "dc4c353d-0b67-4b66-a052-dd8397466d6f") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:08:09.636853 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:09.636667 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:08:09.636853 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:09.636699 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret podName:e59c89a5-bae3-48e6-bdb5-f52094590f45 nodeName:}" failed. No retries permitted until 2026-09-17 07:08:10.636692905 +0000 UTC m=+3.128767688 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret") pod "global-pull-secret-syncer-7jt2z" (UID: "e59c89a5-bae3-48e6-bdb5-f52094590f45") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:08:09.715734 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.715695 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:08:09.737488 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.737449 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg4sk\" (UniqueName: \"kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk\") pod \"network-check-target-vddmm\" (UID: \"3b975ace-eff0-4687-845a-c9607f77ac13\") " pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:09.737679 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:09.737664 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:08:09.737748 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:09.737692 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:08:09.737748 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:09.737702 2566 projected.go:196] Error preparing data for projected volume kube-api-access-hg4sk for pod openshift-network-diagnostics/network-check-target-vddmm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:08:09.737884 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:09.737756 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk podName:3b975ace-eff0-4687-845a-c9607f77ac13 nodeName:}" failed. No retries permitted until 2026-09-17 07:08:10.73773834 +0000 UTC m=+3.229813121 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hg4sk" (UniqueName: "kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk") pod "network-check-target-vddmm" (UID: "3b975ace-eff0-4687-845a-c9607f77ac13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:08:09.989016 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.988864 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 07:03:08 +0000 UTC" deadline="2028-06-06 08:27:54.728371076 +0000 UTC" Sep 17 07:08:09.989016 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:09.988914 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15073h19m44.73946197s" Sep 17 07:08:10.029470 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.029366 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9mr6f" event={"ID":"a9bab8de-9d11-4e20-814c-74e2afb9eb19","Type":"ContainerStarted","Data":"f3b3e614f1bf2915941900128f7f16496be0127e98963c0144ce9eed95690522"} Sep 17 07:08:10.032291 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.032196 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8cl5p" event={"ID":"abd2409d-642b-4c1e-a55c-a4a2c702b553","Type":"ContainerStarted","Data":"d02fd1f7cc243c5db74267911f66ed97b1522880f7c13528a38ccdab0c78fb25"} Sep 17 07:08:10.035083 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.034964 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzr67" event={"ID":"8afb189f-1edd-473f-92dc-f923a60d691e","Type":"ContainerStarted","Data":"9a5c219b5b2c273b5d2e2602d996565d15f72778ebf7a4fd8c7a211fa99ef4e3"} Sep 17 07:08:10.040780 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.040639 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wffc2" event={"ID":"e94051ac-fe49-4338-ab5f-69979421a9cb","Type":"ContainerStarted","Data":"2a2e49cacc0b1eba84c5fafae4f4ba357874cd0acf35dea91652bf6b522b5393"} Sep 17 07:08:10.043208 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.043138 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" event={"ID":"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f","Type":"ContainerStarted","Data":"dc512086dd46bdfe49ac59be09af96dd21fb313114d25b857024dbbd45d00036"} Sep 17 07:08:10.046966 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.046940 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zslxf" event={"ID":"b815c071-1510-4f92-a8be-9d9a49cba65a","Type":"ContainerStarted","Data":"ff602c604818d0d119aa9049582c0e2c56b04654680b36cb4d85073c306bcc02"} Sep 17 07:08:10.053540 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.053483 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal" event={"ID":"f3dcb0e14c4d6a09db4500d0c827116f","Type":"ContainerStarted","Data":"7a4e9b9275c526ba8bb8a2620dd561aeb5d62a5c056d051fc696e0da3cb5b7f5"} Sep 17 07:08:10.056275 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.056198 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nx9rp" event={"ID":"93339a00-d0f0-4f0d-9449-bbb4566c2b93","Type":"ContainerStarted","Data":"85ff539ba63caf9d6b45df8e8b65692660fd492da77a87a14be139cf16a879de"} Sep 17 07:08:10.060792 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.060739 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" event={"ID":"3953be2c-e8eb-4172-88fa-8d04babc59fd","Type":"ContainerStarted","Data":"bc43acc550283099c132f7f310b5fa8088613edfdb1a61c78b2ad006ab59542e"} Sep 17 07:08:10.065752 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.065722 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kvftr" event={"ID":"7a9610b5-26e5-44c9-a253-ea18eaec24af","Type":"ContainerStarted","Data":"22c4e4b3ea05d975f67fe8be31147d1603e45786b44f584abfcdae23a0d605c6"} Sep 17 07:08:10.068433 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.068407 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-90.ec2.internal" event={"ID":"b5d2ed14de66e6a088f1d600ae6da75a","Type":"ContainerStarted","Data":"9a380f29128b046944497109a3d76bf91b88b68fbe8efcc56ec3adbe138f4291"} Sep 17 07:08:10.646902 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.646122 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret\") pod \"global-pull-secret-syncer-7jt2z\" (UID: \"e59c89a5-bae3-48e6-bdb5-f52094590f45\") " pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:10.646902 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.646197 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs\") pod \"network-metrics-daemon-z2l2s\" (UID: \"dc4c353d-0b67-4b66-a052-dd8397466d6f\") " pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:10.646902 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:10.646325 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:08:10.646902 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:10.646389 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs podName:dc4c353d-0b67-4b66-a052-dd8397466d6f nodeName:}" failed. No retries permitted until 2026-09-17 07:08:12.646369565 +0000 UTC m=+5.138444350 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs") pod "network-metrics-daemon-z2l2s" (UID: "dc4c353d-0b67-4b66-a052-dd8397466d6f") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:08:10.646902 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:10.646816 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:08:10.646902 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:10.646869 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret podName:e59c89a5-bae3-48e6-bdb5-f52094590f45 nodeName:}" failed. No retries permitted until 2026-09-17 07:08:12.646854635 +0000 UTC m=+5.138929423 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret") pod "global-pull-secret-syncer-7jt2z" (UID: "e59c89a5-bae3-48e6-bdb5-f52094590f45") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:08:10.738384 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.738355 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:08:10.748451 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.746652 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg4sk\" (UniqueName: \"kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk\") pod \"network-check-target-vddmm\" (UID: \"3b975ace-eff0-4687-845a-c9607f77ac13\") " pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:10.748451 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:10.746836 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:08:10.748451 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:10.746859 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:08:10.748451 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:10.746872 2566 projected.go:196] Error preparing data for projected volume kube-api-access-hg4sk for pod openshift-network-diagnostics/network-check-target-vddmm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:08:10.748451 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:10.746934 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk podName:3b975ace-eff0-4687-845a-c9607f77ac13 nodeName:}" failed. No retries permitted until 2026-09-17 07:08:12.746916129 +0000 UTC m=+5.238990900 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hg4sk" (UniqueName: "kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk") pod "network-check-target-vddmm" (UID: "3b975ace-eff0-4687-845a-c9607f77ac13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:08:10.863664 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.863557 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:08:10.989830 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.989702 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 07:03:08 +0000 UTC" deadline="2028-04-07 18:20:08.904095204 +0000 UTC" Sep 17 07:08:10.989830 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:10.989767 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13643h11m57.914332413s" Sep 17 07:08:11.018302 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:11.018270 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:11.018480 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:11.018395 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:11.018839 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:11.018819 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:11.019040 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:11.018935 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:11.019040 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:11.019005 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:11.019164 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:11.019077 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:12.663634 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:12.663546 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs\") pod \"network-metrics-daemon-z2l2s\" (UID: \"dc4c353d-0b67-4b66-a052-dd8397466d6f\") " pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:12.664121 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:12.663634 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret\") pod \"global-pull-secret-syncer-7jt2z\" (UID: \"e59c89a5-bae3-48e6-bdb5-f52094590f45\") " pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:12.664121 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:12.663777 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:08:12.664121 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:12.663852 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret podName:e59c89a5-bae3-48e6-bdb5-f52094590f45 nodeName:}" failed. No retries permitted until 2026-09-17 07:08:16.663834183 +0000 UTC m=+9.155908960 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret") pod "global-pull-secret-syncer-7jt2z" (UID: "e59c89a5-bae3-48e6-bdb5-f52094590f45") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:08:12.664286 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:12.664255 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:08:12.664335 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:12.664330 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs podName:dc4c353d-0b67-4b66-a052-dd8397466d6f nodeName:}" failed. No retries permitted until 2026-09-17 07:08:16.664314133 +0000 UTC m=+9.156388904 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs") pod "network-metrics-daemon-z2l2s" (UID: "dc4c353d-0b67-4b66-a052-dd8397466d6f") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:08:12.764475 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:12.764427 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg4sk\" (UniqueName: \"kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk\") pod \"network-check-target-vddmm\" (UID: \"3b975ace-eff0-4687-845a-c9607f77ac13\") " pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:12.764693 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:12.764666 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:08:12.764766 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:12.764691 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:08:12.764766 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:12.764713 2566 projected.go:196] Error preparing data for projected volume kube-api-access-hg4sk for pod openshift-network-diagnostics/network-check-target-vddmm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:08:12.764917 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:12.764770 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk podName:3b975ace-eff0-4687-845a-c9607f77ac13 nodeName:}" failed. No retries permitted until 2026-09-17 07:08:16.764753665 +0000 UTC m=+9.256828438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hg4sk" (UniqueName: "kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk") pod "network-check-target-vddmm" (UID: "3b975ace-eff0-4687-845a-c9607f77ac13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:08:13.018249 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:13.018171 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:13.018409 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:13.018303 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:13.018680 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:13.018662 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:13.018805 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:13.018768 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:13.018879 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:13.018832 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:13.018975 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:13.018957 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:15.018378 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:15.018272 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:15.018928 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:15.018396 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:15.018928 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:15.018433 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:15.018928 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:15.018455 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:15.018928 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:15.018536 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:15.019140 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:15.018927 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:16.697763 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:16.697000 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs\") pod \"network-metrics-daemon-z2l2s\" (UID: \"dc4c353d-0b67-4b66-a052-dd8397466d6f\") " pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:16.697763 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:16.697086 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret\") pod \"global-pull-secret-syncer-7jt2z\" (UID: \"e59c89a5-bae3-48e6-bdb5-f52094590f45\") " pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:16.697763 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:16.697227 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:08:16.697763 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:16.697290 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret podName:e59c89a5-bae3-48e6-bdb5-f52094590f45 nodeName:}" failed. No retries permitted until 2026-09-17 07:08:24.697272087 +0000 UTC m=+17.189346860 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret") pod "global-pull-secret-syncer-7jt2z" (UID: "e59c89a5-bae3-48e6-bdb5-f52094590f45") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:08:16.697763 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:16.697687 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:08:16.697763 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:16.697732 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs podName:dc4c353d-0b67-4b66-a052-dd8397466d6f nodeName:}" failed. No retries permitted until 2026-09-17 07:08:24.69771728 +0000 UTC m=+17.189792053 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs") pod "network-metrics-daemon-z2l2s" (UID: "dc4c353d-0b67-4b66-a052-dd8397466d6f") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:08:16.798485 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:16.797816 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg4sk\" (UniqueName: \"kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk\") pod \"network-check-target-vddmm\" (UID: \"3b975ace-eff0-4687-845a-c9607f77ac13\") " pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:16.798485 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:16.798019 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:08:16.798485 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:16.798040 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:08:16.798485 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:16.798054 2566 projected.go:196] Error preparing data for projected volume kube-api-access-hg4sk for pod openshift-network-diagnostics/network-check-target-vddmm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:08:16.798485 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:16.798111 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk podName:3b975ace-eff0-4687-845a-c9607f77ac13 nodeName:}" failed. No retries permitted until 2026-09-17 07:08:24.798093437 +0000 UTC m=+17.290168214 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hg4sk" (UniqueName: "kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk") pod "network-check-target-vddmm" (UID: "3b975ace-eff0-4687-845a-c9607f77ac13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:08:17.018473 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:17.018434 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:17.018473 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:17.018460 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:17.018706 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:17.018444 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:17.018706 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:17.018565 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:17.018706 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:17.018653 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:17.018876 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:17.018736 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:19.017466 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:19.017427 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:19.017931 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:19.017428 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:19.017931 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:19.017548 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:19.017931 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:19.017429 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:19.017931 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:19.017642 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:19.017931 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:19.017776 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:21.017684 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:21.017607 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:21.017684 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:21.017675 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:21.018218 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:21.017628 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:21.018218 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:21.017773 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:21.018218 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:21.017887 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:21.018218 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:21.017969 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:23.017615 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:23.017580 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:23.018103 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:23.017580 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:23.018103 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:23.017710 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:23.018103 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:23.017588 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:23.018103 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:23.017797 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:23.018103 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:23.017880 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:24.754130 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:24.754088 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret\") pod \"global-pull-secret-syncer-7jt2z\" (UID: \"e59c89a5-bae3-48e6-bdb5-f52094590f45\") " pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:24.754591 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:24.754159 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs\") pod \"network-metrics-daemon-z2l2s\" (UID: \"dc4c353d-0b67-4b66-a052-dd8397466d6f\") " pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:24.754591 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:24.754272 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:08:24.754591 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:24.754275 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:08:24.754591 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:24.754345 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret podName:e59c89a5-bae3-48e6-bdb5-f52094590f45 nodeName:}" failed. No retries permitted until 2026-09-17 07:08:40.754324804 +0000 UTC m=+33.246399590 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret") pod "global-pull-secret-syncer-7jt2z" (UID: "e59c89a5-bae3-48e6-bdb5-f52094590f45") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:08:24.754591 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:24.754366 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs podName:dc4c353d-0b67-4b66-a052-dd8397466d6f nodeName:}" failed. No retries permitted until 2026-09-17 07:08:40.754354079 +0000 UTC m=+33.246428856 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs") pod "network-metrics-daemon-z2l2s" (UID: "dc4c353d-0b67-4b66-a052-dd8397466d6f") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:08:24.854971 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:24.854936 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg4sk\" (UniqueName: \"kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk\") pod \"network-check-target-vddmm\" (UID: \"3b975ace-eff0-4687-845a-c9607f77ac13\") " pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:24.855132 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:24.855081 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:08:24.855132 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:24.855096 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:08:24.855132 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:24.855105 2566 projected.go:196] Error preparing data for projected volume kube-api-access-hg4sk for pod openshift-network-diagnostics/network-check-target-vddmm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:08:24.855241 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:24.855155 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk podName:3b975ace-eff0-4687-845a-c9607f77ac13 nodeName:}" failed. No retries permitted until 2026-09-17 07:08:40.855141011 +0000 UTC m=+33.347215785 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hg4sk" (UniqueName: "kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk") pod "network-check-target-vddmm" (UID: "3b975ace-eff0-4687-845a-c9607f77ac13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:08:25.017965 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:25.017920 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:25.017965 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:25.017963 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:25.018168 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:25.017963 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:25.018168 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:25.018057 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:25.018168 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:25.018156 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:25.018299 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:25.018208 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:27.017962 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:27.017922 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:27.018493 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:27.017964 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:27.018493 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:27.017964 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:27.018493 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:27.018054 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:27.018493 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:27.018137 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:27.018493 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:27.018217 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:29.018348 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.018016 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:29.018974 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.018065 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:29.018974 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:29.018429 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:29.018974 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.018067 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:29.018974 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:29.018549 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:29.018974 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:29.018623 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:29.112272 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.112194 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nx9rp" event={"ID":"93339a00-d0f0-4f0d-9449-bbb4566c2b93","Type":"ContainerStarted","Data":"dbc637467908ceb65a015990d4e131e9687f761ff1d6232cd24bb6d32326bc83"} Sep 17 07:08:29.115082 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.115060 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:08:29.115448 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.115403 2566 generic.go:356] "Generic (PLEG): container finished" podID="3953be2c-e8eb-4172-88fa-8d04babc59fd" containerID="38f919b69a17a32dae11982a21ca247b1eadcd595e16243d1ddaec962908b343" exitCode=1 Sep 17 07:08:29.115530 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.115475 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" event={"ID":"3953be2c-e8eb-4172-88fa-8d04babc59fd","Type":"ContainerStarted","Data":"3b3e7ad08be06e8e56cbfb248f35022b3db19edea33bb8a3090da2339fa878ee"} Sep 17 07:08:29.115598 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.115533 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" event={"ID":"3953be2c-e8eb-4172-88fa-8d04babc59fd","Type":"ContainerStarted","Data":"9cb73e61c6a6e117f2194aa2ed14e23fbbed4f2ff657d6b3eaa09bf2404559e9"} Sep 17 07:08:29.115598 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.115567 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" event={"ID":"3953be2c-e8eb-4172-88fa-8d04babc59fd","Type":"ContainerStarted","Data":"771df65d1ebb8393a978bec02a7c150b0c6877b81f2e9d686c3cac7480eebe31"} Sep 17 07:08:29.115675 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.115612 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" event={"ID":"3953be2c-e8eb-4172-88fa-8d04babc59fd","Type":"ContainerStarted","Data":"85f3318b1f7500ee1118272a9054054a11992946910288d69990f970b98d9d4d"} Sep 17 07:08:29.115675 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.115657 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" event={"ID":"3953be2c-e8eb-4172-88fa-8d04babc59fd","Type":"ContainerDied","Data":"38f919b69a17a32dae11982a21ca247b1eadcd595e16243d1ddaec962908b343"} Sep 17 07:08:29.115747 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.115717 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" event={"ID":"3953be2c-e8eb-4172-88fa-8d04babc59fd","Type":"ContainerStarted","Data":"e3b39040daa7e8b2f5369e1e572c9063115a75b1eca3680d50a428b72b619647"} Sep 17 07:08:29.116942 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.116916 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kvftr" event={"ID":"7a9610b5-26e5-44c9-a253-ea18eaec24af","Type":"ContainerStarted","Data":"65f3cd455a0975bc8357e12c3a6e02b0db7719247f0187aaf638966d7bd07e52"} Sep 17 07:08:29.118314 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.118294 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-90.ec2.internal" event={"ID":"b5d2ed14de66e6a088f1d600ae6da75a","Type":"ContainerStarted","Data":"8e9e4fadb92b23f67abf8ea8c1977dfed025ae5de7f1545730b7a5471f9e594a"} Sep 17 07:08:29.119579 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.119562 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9mr6f" event={"ID":"a9bab8de-9d11-4e20-814c-74e2afb9eb19","Type":"ContainerStarted","Data":"e6cf79bf7979b78adea4315bf6feaba2db9e47a346a068cdb1b54d514c77125d"} Sep 17 07:08:29.120931 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.120911 2566 generic.go:356] "Generic (PLEG): container finished" podID="8afb189f-1edd-473f-92dc-f923a60d691e" containerID="5c3fc9cc16efec55fe919fc0b66a8a04b4434470afe6656b9095fe38baee0894" exitCode=0 Sep 17 07:08:29.121016 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.120988 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzr67" event={"ID":"8afb189f-1edd-473f-92dc-f923a60d691e","Type":"ContainerDied","Data":"5c3fc9cc16efec55fe919fc0b66a8a04b4434470afe6656b9095fe38baee0894"} Sep 17 07:08:29.122427 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.122409 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wffc2" event={"ID":"e94051ac-fe49-4338-ab5f-69979421a9cb","Type":"ContainerStarted","Data":"8f13cc05bf79d2220bc0b55ba036d3fbace2d06032ca8271af2a9a3c2ece4227"} Sep 17 07:08:29.123925 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.123908 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" event={"ID":"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f","Type":"ContainerStarted","Data":"eb611166290b6662e37393965b8a5abb8fb2faf55c7eaeb5f2964b995c4ce683"} Sep 17 07:08:29.125280 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.125262 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zslxf" event={"ID":"b815c071-1510-4f92-a8be-9d9a49cba65a","Type":"ContainerStarted","Data":"51cba0e2c81f86de6977239855da96448ea22b076e55b83ad4b18f1cb01e1ae7"} Sep 17 07:08:29.126667 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.126646 2566 generic.go:356] "Generic (PLEG): container finished" podID="f3dcb0e14c4d6a09db4500d0c827116f" containerID="2ee4b2b81bbe9e13c9fceec87ec848efe74151212b8d952259d63d04eae841b6" exitCode=0 Sep 17 07:08:29.126739 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.126678 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal" event={"ID":"f3dcb0e14c4d6a09db4500d0c827116f","Type":"ContainerDied","Data":"2ee4b2b81bbe9e13c9fceec87ec848efe74151212b8d952259d63d04eae841b6"} Sep 17 07:08:29.143416 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.143373 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nx9rp" podStartSLOduration=1.994082575 podStartE2EDuration="21.143361852s" totalImagesPullingTime="19.149279277s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:08:29.143118746 +0000 UTC m=+21.635193551" watchObservedRunningTime="2026-09-17 07:08:29.143361852 +0000 UTC m=+21.635436644" Sep 17 07:08:29.175943 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.175899 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-90.ec2.internal" podStartSLOduration=21.175879839 podStartE2EDuration="21.175879839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:08:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:08:29.175660874 +0000 UTC m=+21.667735679" watchObservedRunningTime="2026-09-17 07:08:29.175879839 +0000 UTC m=+21.667954635" Sep 17 07:08:29.236550 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.236496 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wffc2" podStartSLOduration=2.444558986 podStartE2EDuration="21.236476622s" totalImagesPullingTime="18.791917636s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:08:29.20026815 +0000 UTC m=+21.692342944" watchObservedRunningTime="2026-09-17 07:08:29.236476622 +0000 UTC m=+21.728551417" Sep 17 07:08:29.344611 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.344558 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zslxf" podStartSLOduration=2.328951346 podStartE2EDuration="21.344540573s" totalImagesPullingTime="19.015589227s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:08:29.315954349 +0000 UTC m=+21.808029143" watchObservedRunningTime="2026-09-17 07:08:29.344540573 +0000 UTC m=+21.836615366" Sep 17 07:08:29.372600 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:29.372515 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kvftr" podStartSLOduration=2.545058016 podStartE2EDuration="21.372499982s" totalImagesPullingTime="18.827441966s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:08:29.345487177 +0000 UTC m=+21.837561970" watchObservedRunningTime="2026-09-17 07:08:29.372499982 +0000 UTC m=+21.864574775" Sep 17 07:08:30.101747 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:30.101721 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 17 07:08:30.130384 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:30.130354 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8cl5p" event={"ID":"abd2409d-642b-4c1e-a55c-a4a2c702b553","Type":"ContainerStarted","Data":"f82f1ac876937d64b804cc6fd55fa76875e2b96d8b60c25c402b6a4e62c3f4c0"} Sep 17 07:08:30.132170 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:30.132148 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" event={"ID":"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f","Type":"ContainerStarted","Data":"d3af7a5bbbe96506dbf5d696a7cc2678a74000132cbf838a4f75aa9fb83fbb4e"} Sep 17 07:08:30.133836 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:30.133803 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal" event={"ID":"f3dcb0e14c4d6a09db4500d0c827116f","Type":"ContainerStarted","Data":"b7901a9da16999a26287cb2d4200ad2711a5ad511fe117559a26e619924ecb42"} Sep 17 07:08:30.155030 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:30.154985 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9mr6f" podStartSLOduration=11.043734453 podStartE2EDuration="22.154969695s" totalImagesPullingTime="11.111235242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:08:29.375819096 +0000 UTC m=+21.867893880" watchObservedRunningTime="2026-09-17 07:08:30.154969695 +0000 UTC m=+22.647044487" Sep 17 07:08:30.174477 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:30.174429 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8cl5p" podStartSLOduration=7.005450334 podStartE2EDuration="22.174413111s" totalImagesPullingTime="15.168962777s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:08:30.1557242 +0000 UTC m=+22.647798992" watchObservedRunningTime="2026-09-17 07:08:30.174413111 +0000 UTC m=+22.666487904" Sep 17 07:08:30.174808 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:30.174769 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-90.ec2.internal" podStartSLOduration=22.174761466 podStartE2EDuration="22.174761466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:08:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:08:30.174502729 +0000 UTC m=+22.666577522" watchObservedRunningTime="2026-09-17 07:08:30.174761466 +0000 UTC m=+22.666836259" Sep 17 07:08:31.006903 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:31.006778 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T07:08:30.101740467Z","UUID":"7af6b2c3-2fcd-43d7-b9d3-a5457d39ebbb","Handler":null,"Name":"","Endpoint":""} Sep 17 07:08:31.008739 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:31.008717 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 17 07:08:31.008900 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:31.008752 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 17 07:08:31.017586 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:31.017565 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:31.017735 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:31.017565 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:31.017735 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:31.017691 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:31.017912 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:31.017806 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:31.017912 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:31.017565 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:31.017912 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:31.017897 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:31.140140 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:31.140120 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:08:31.140545 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:31.140520 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" event={"ID":"3953be2c-e8eb-4172-88fa-8d04babc59fd","Type":"ContainerStarted","Data":"0b2614f8d76cb532d144e77ee6564402743de55aa6d6ecebca13d06e58b8a38a"} Sep 17 07:08:32.145317 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:32.145274 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" event={"ID":"9aed1d6e-0d13-4262-a9ad-48fbcd4e8d5f","Type":"ContainerStarted","Data":"2d42042115c5a28a3ccd7a9f52535a4bdb2958775e2e071443bc8b764ccf2be8"} Sep 17 07:08:32.168867 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:32.168628 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nl2t7" podStartSLOduration=3.222364128 podStartE2EDuration="24.168610283s" totalImagesPullingTime="20.946246155s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:08:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:08:32.168473254 +0000 UTC m=+24.660548048" watchObservedRunningTime="2026-09-17 07:08:32.168610283 +0000 UTC m=+24.660685077" Sep 17 07:08:33.017662 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:33.017623 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:33.017662 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:33.017656 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:33.017902 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:33.017664 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:33.017902 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:33.017769 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:33.018007 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:33.017907 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:33.018053 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:33.018004 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:33.293171 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:33.293133 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9mr6f" Sep 17 07:08:33.293995 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:33.293973 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9mr6f" Sep 17 07:08:34.048020 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:34.047989 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:34.149843 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:34.149818 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9mr6f" Sep 17 07:08:34.150517 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:34.150390 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9mr6f" Sep 17 07:08:34.172940 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:34.172912 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:34.285601 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:34.285555 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:34.539864 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:34.539829 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:35.018191 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:35.018142 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:35.018191 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:35.018182 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:35.018421 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:35.018144 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:35.018421 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:35.018245 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:35.018421 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:35.018346 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:35.018421 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:35.018401 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:35.153753 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:35.153724 2566 generic.go:356] "Generic (PLEG): container finished" podID="8afb189f-1edd-473f-92dc-f923a60d691e" containerID="b6a29092d5364be641005e9ac44f1af586b91c16c39b170a075e48f47ec338d4" exitCode=0 Sep 17 07:08:35.153907 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:35.153817 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzr67" event={"ID":"8afb189f-1edd-473f-92dc-f923a60d691e","Type":"ContainerDied","Data":"b6a29092d5364be641005e9ac44f1af586b91c16c39b170a075e48f47ec338d4"} Sep 17 07:08:35.157524 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:35.157510 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:08:35.157906 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:35.157887 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" event={"ID":"3953be2c-e8eb-4172-88fa-8d04babc59fd","Type":"ContainerStarted","Data":"7ff21e4043c17bf0a258d6740bb56bf1cd02ef999b67e6929654909074f85adb"} Sep 17 07:08:35.158316 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:35.158302 2566 scope.go:117] "RemoveContainer" containerID="38f919b69a17a32dae11982a21ca247b1eadcd595e16243d1ddaec962908b343" Sep 17 07:08:35.182831 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:35.182804 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:35.271851 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:35.271628 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:35.362558 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:35.362519 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:35.436520 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:35.436481 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:36.068506 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:36.068456 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7jt2z"] Sep 17 07:08:36.069267 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:36.068571 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:36.069267 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:36.068683 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:36.074474 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:36.074415 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-z2l2s"] Sep 17 07:08:36.074628 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:36.074517 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:36.074693 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:36.074629 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:36.086674 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:36.086642 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vddmm"] Sep 17 07:08:36.086862 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:36.086756 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:36.086932 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:36.086873 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:36.167166 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:36.167139 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:08:36.167569 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:36.167546 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" event={"ID":"3953be2c-e8eb-4172-88fa-8d04babc59fd","Type":"ContainerStarted","Data":"8743a8b42cd3ce810fca86dbf31d6becfcb55e7d976b4572959f3e5a1f9a16bd"} Sep 17 07:08:36.168209 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:36.167796 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:36.168209 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:36.167966 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:36.168209 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:36.168045 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:36.185846 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:36.185819 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:36.185989 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:36.185943 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:08:36.215763 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:36.215588 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" podStartSLOduration=9.188793044 podStartE2EDuration="28.21557177s" totalImagesPullingTime="19.026778726s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:08:36.210905068 +0000 UTC m=+28.702979861" watchObservedRunningTime="2026-09-17 07:08:36.21557177 +0000 UTC m=+28.707646563" Sep 17 07:08:37.040286 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:37.040253 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:37.172219 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:37.172183 2566 generic.go:356] "Generic (PLEG): container finished" podID="8afb189f-1edd-473f-92dc-f923a60d691e" containerID="5a57d9313545b9f8187b25b75c64393ae0a6256c90849e2f1ae865c14d9ac7d6" exitCode=0 Sep 17 07:08:37.172829 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:37.172261 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzr67" event={"ID":"8afb189f-1edd-473f-92dc-f923a60d691e","Type":"ContainerDied","Data":"5a57d9313545b9f8187b25b75c64393ae0a6256c90849e2f1ae865c14d9ac7d6"} Sep 17 07:08:37.178711 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:37.178677 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:38.019293 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:38.019061 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:38.019457 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:38.019125 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:38.019457 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:38.019394 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:38.019457 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:38.019150 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:38.019457 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:38.019446 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:38.019614 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:38.019501 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:39.180216 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:39.180174 2566 generic.go:356] "Generic (PLEG): container finished" podID="8afb189f-1edd-473f-92dc-f923a60d691e" containerID="73509fa7f64f3564fed631ab06e392af3e7f6f363bdd159a6cb7e3f59d0636ef" exitCode=0 Sep 17 07:08:39.180581 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:39.180231 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzr67" event={"ID":"8afb189f-1edd-473f-92dc-f923a60d691e","Type":"ContainerDied","Data":"73509fa7f64f3564fed631ab06e392af3e7f6f363bdd159a6cb7e3f59d0636ef"} Sep 17 07:08:39.369065 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:39.369029 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:40.017528 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:40.017491 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:40.017528 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:40.017525 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:40.017772 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:40.017523 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:40.017772 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:40.017622 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z2l2s" podUID="dc4c353d-0b67-4b66-a052-dd8397466d6f" Sep 17 07:08:40.017892 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:40.017766 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7jt2z" podUID="e59c89a5-bae3-48e6-bdb5-f52094590f45" Sep 17 07:08:40.018550 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:40.018500 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vddmm" podUID="3b975ace-eff0-4687-845a-c9607f77ac13" Sep 17 07:08:40.765162 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:40.765123 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret\") pod \"global-pull-secret-syncer-7jt2z\" (UID: \"e59c89a5-bae3-48e6-bdb5-f52094590f45\") " pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:40.765551 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:40.765176 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs\") pod \"network-metrics-daemon-z2l2s\" (UID: \"dc4c353d-0b67-4b66-a052-dd8397466d6f\") " pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:40.765551 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:40.765268 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:08:40.765551 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:40.765269 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:08:40.765551 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:40.765318 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs podName:dc4c353d-0b67-4b66-a052-dd8397466d6f nodeName:}" failed. No retries permitted until 2026-09-17 07:09:12.765304087 +0000 UTC m=+65.257378858 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs") pod "network-metrics-daemon-z2l2s" (UID: "dc4c353d-0b67-4b66-a052-dd8397466d6f") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:08:40.765551 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:40.765330 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret podName:e59c89a5-bae3-48e6-bdb5-f52094590f45 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:12.765324727 +0000 UTC m=+65.257399498 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret") pod "global-pull-secret-syncer-7jt2z" (UID: "e59c89a5-bae3-48e6-bdb5-f52094590f45") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:08:40.866608 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:40.866567 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg4sk\" (UniqueName: \"kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk\") pod \"network-check-target-vddmm\" (UID: \"3b975ace-eff0-4687-845a-c9607f77ac13\") " pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:40.866770 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:40.866719 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:08:40.866770 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:40.866737 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:08:40.866770 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:40.866749 2566 projected.go:196] Error preparing data for projected volume kube-api-access-hg4sk for pod openshift-network-diagnostics/network-check-target-vddmm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:08:40.866939 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:40.866822 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk podName:3b975ace-eff0-4687-845a-c9607f77ac13 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:12.86680373 +0000 UTC m=+65.358878503 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hg4sk" (UniqueName: "kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk") pod "network-check-target-vddmm" (UID: "3b975ace-eff0-4687-845a-c9607f77ac13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:08:41.256262 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.256225 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-90.ec2.internal" event="NodeReady" Sep 17 07:08:41.256418 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.256377 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 17 07:08:41.356236 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.356192 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2n76n"] Sep 17 07:08:41.399875 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.399842 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-79g8v"] Sep 17 07:08:41.400049 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.399973 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2n76n" Sep 17 07:08:41.410108 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.410065 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 17 07:08:41.410108 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.410078 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 17 07:08:41.410389 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.410112 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qh9gr\"" Sep 17 07:08:41.410389 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.410098 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 17 07:08:41.413648 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.413620 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-79g8v"] Sep 17 07:08:41.413741 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.413663 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2n76n"] Sep 17 07:08:41.413794 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.413746 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:41.418303 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.418284 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wq5pm\"" Sep 17 07:08:41.418692 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.418598 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 17 07:08:41.419055 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.419000 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 17 07:08:41.470667 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.470638 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f70632fa-c15c-4beb-8c27-2b4316baafd5-cert\") pod \"ingress-canary-2n76n\" (UID: \"f70632fa-c15c-4beb-8c27-2b4316baafd5\") " pod="openshift-ingress-canary/ingress-canary-2n76n" Sep 17 07:08:41.470873 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.470703 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a647a2aa-ba1d-4809-9639-dddcba088b0e-tmp-dir\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:41.470873 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.470731 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb4g8\" (UniqueName: \"kubernetes.io/projected/f70632fa-c15c-4beb-8c27-2b4316baafd5-kube-api-access-pb4g8\") pod \"ingress-canary-2n76n\" (UID: \"f70632fa-c15c-4beb-8c27-2b4316baafd5\") " pod="openshift-ingress-canary/ingress-canary-2n76n" Sep 17 07:08:41.470873 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.470823 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a647a2aa-ba1d-4809-9639-dddcba088b0e-config-volume\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:41.470873 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.470848 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9jvw\" (UniqueName: \"kubernetes.io/projected/a647a2aa-ba1d-4809-9639-dddcba088b0e-kube-api-access-h9jvw\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:41.471074 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.470913 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:41.572112 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.572078 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a647a2aa-ba1d-4809-9639-dddcba088b0e-config-volume\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:41.572112 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.572113 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9jvw\" (UniqueName: \"kubernetes.io/projected/a647a2aa-ba1d-4809-9639-dddcba088b0e-kube-api-access-h9jvw\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:41.572304 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.572160 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:41.572304 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.572194 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f70632fa-c15c-4beb-8c27-2b4316baafd5-cert\") pod \"ingress-canary-2n76n\" (UID: \"f70632fa-c15c-4beb-8c27-2b4316baafd5\") " pod="openshift-ingress-canary/ingress-canary-2n76n" Sep 17 07:08:41.572304 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.572208 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a647a2aa-ba1d-4809-9639-dddcba088b0e-tmp-dir\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:41.572304 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.572229 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb4g8\" (UniqueName: \"kubernetes.io/projected/f70632fa-c15c-4beb-8c27-2b4316baafd5-kube-api-access-pb4g8\") pod \"ingress-canary-2n76n\" (UID: \"f70632fa-c15c-4beb-8c27-2b4316baafd5\") " pod="openshift-ingress-canary/ingress-canary-2n76n" Sep 17 07:08:41.572485 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:41.572394 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:08:41.572485 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:41.572463 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls podName:a647a2aa-ba1d-4809-9639-dddcba088b0e nodeName:}" failed. No retries permitted until 2026-09-17 07:08:42.072442296 +0000 UTC m=+34.564517091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls") pod "dns-default-79g8v" (UID: "a647a2aa-ba1d-4809-9639-dddcba088b0e") : secret "dns-default-metrics-tls" not found Sep 17 07:08:41.572584 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.572520 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a647a2aa-ba1d-4809-9639-dddcba088b0e-tmp-dir\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:41.572749 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.572730 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a647a2aa-ba1d-4809-9639-dddcba088b0e-config-volume\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:41.576194 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.576173 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f70632fa-c15c-4beb-8c27-2b4316baafd5-cert\") pod \"ingress-canary-2n76n\" (UID: \"f70632fa-c15c-4beb-8c27-2b4316baafd5\") " pod="openshift-ingress-canary/ingress-canary-2n76n" Sep 17 07:08:41.595335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.595304 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb4g8\" (UniqueName: \"kubernetes.io/projected/f70632fa-c15c-4beb-8c27-2b4316baafd5-kube-api-access-pb4g8\") pod \"ingress-canary-2n76n\" (UID: \"f70632fa-c15c-4beb-8c27-2b4316baafd5\") " pod="openshift-ingress-canary/ingress-canary-2n76n" Sep 17 07:08:41.603760 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.603737 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9jvw\" (UniqueName: \"kubernetes.io/projected/a647a2aa-ba1d-4809-9639-dddcba088b0e-kube-api-access-h9jvw\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:41.710449 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.710414 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2n76n" Sep 17 07:08:41.900470 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:41.900395 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:42.017563 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:42.017530 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:08:42.017911 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:42.017530 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:08:42.017911 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:42.017530 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:08:42.020166 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:42.020143 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 07:08:42.021332 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:42.021294 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2n76n"] Sep 17 07:08:42.022967 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:42.022951 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g2nsv\"" Sep 17 07:08:42.023701 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:42.023673 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2f8p7\"" Sep 17 07:08:42.023915 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:08:42.023893 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf70632fa_c15c_4beb_8c27_2b4316baafd5.slice/crio-ecbe8dbb948a596fe60a2f3b9a5f7fa31e02c4aaedc4f0258690e0599aca5dca WatchSource:0}: Error finding container ecbe8dbb948a596fe60a2f3b9a5f7fa31e02c4aaedc4f0258690e0599aca5dca: Status 404 returned error can't find the container with id ecbe8dbb948a596fe60a2f3b9a5f7fa31e02c4aaedc4f0258690e0599aca5dca Sep 17 07:08:42.024531 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:42.024513 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 07:08:42.025953 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:42.025934 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 07:08:42.026183 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:42.026168 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 07:08:42.076938 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:42.076896 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:42.077129 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:42.077054 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:08:42.077129 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:42.077123 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls podName:a647a2aa-ba1d-4809-9639-dddcba088b0e nodeName:}" failed. No retries permitted until 2026-09-17 07:08:43.077104056 +0000 UTC m=+35.569178833 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls") pod "dns-default-79g8v" (UID: "a647a2aa-ba1d-4809-9639-dddcba088b0e") : secret "dns-default-metrics-tls" not found Sep 17 07:08:42.188711 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:42.188624 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2n76n" event={"ID":"f70632fa-c15c-4beb-8c27-2b4316baafd5","Type":"ContainerStarted","Data":"ecbe8dbb948a596fe60a2f3b9a5f7fa31e02c4aaedc4f0258690e0599aca5dca"} Sep 17 07:08:43.084529 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:43.084289 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:43.085132 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:43.084444 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:08:43.085132 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:43.084679 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls podName:a647a2aa-ba1d-4809-9639-dddcba088b0e nodeName:}" failed. No retries permitted until 2026-09-17 07:08:45.084656647 +0000 UTC m=+37.576731430 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls") pod "dns-default-79g8v" (UID: "a647a2aa-ba1d-4809-9639-dddcba088b0e") : secret "dns-default-metrics-tls" not found Sep 17 07:08:45.098921 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:45.098878 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:45.099383 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:45.099049 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:08:45.099383 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:45.099123 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls podName:a647a2aa-ba1d-4809-9639-dddcba088b0e nodeName:}" failed. No retries permitted until 2026-09-17 07:08:49.099104307 +0000 UTC m=+41.591179099 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls") pod "dns-default-79g8v" (UID: "a647a2aa-ba1d-4809-9639-dddcba088b0e") : secret "dns-default-metrics-tls" not found Sep 17 07:08:46.179746 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:46.179706 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:46.200917 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:46.200880 2566 generic.go:356] "Generic (PLEG): container finished" podID="8afb189f-1edd-473f-92dc-f923a60d691e" containerID="1e6a173e826fb7da5a39e7d145a9db604d5dd202f2062c50b989c12017d2a654" exitCode=0 Sep 17 07:08:46.201090 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:46.200967 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzr67" event={"ID":"8afb189f-1edd-473f-92dc-f923a60d691e","Type":"ContainerDied","Data":"1e6a173e826fb7da5a39e7d145a9db604d5dd202f2062c50b989c12017d2a654"} Sep 17 07:08:46.202391 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:46.202371 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2n76n" event={"ID":"f70632fa-c15c-4beb-8c27-2b4316baafd5","Type":"ContainerStarted","Data":"201a0dc4307be78ea4a20d0f34477746de2369dffe9bfc78e80937ad9322442e"} Sep 17 07:08:46.248463 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:46.248415 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2n76n" podStartSLOduration=1.834477962 podStartE2EDuration="5.248398934s" totalImagesPullingTime="3.413920972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:08:46.247947962 +0000 UTC m=+38.740022756" watchObservedRunningTime="2026-09-17 07:08:46.248398934 +0000 UTC m=+38.740473727" Sep 17 07:08:46.640857 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:46.640827 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:46.826292 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:46.826261 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:47.103207 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:47.103178 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:47.205330 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:47.205289 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:47.207687 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:47.207661 2566 generic.go:356] "Generic (PLEG): container finished" podID="8afb189f-1edd-473f-92dc-f923a60d691e" containerID="cc34e9030f6b408302146726d27686c4752cbf6a7576b4cc0c000eb59748608a" exitCode=0 Sep 17 07:08:47.207820 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:47.207691 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzr67" event={"ID":"8afb189f-1edd-473f-92dc-f923a60d691e","Type":"ContainerDied","Data":"cc34e9030f6b408302146726d27686c4752cbf6a7576b4cc0c000eb59748608a"} Sep 17 07:08:48.213260 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:48.213219 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzr67" event={"ID":"8afb189f-1edd-473f-92dc-f923a60d691e","Type":"ContainerStarted","Data":"7320fe3bad04470e36f83e126a7eaaa41acac7bd168e9fb92a2c8b8fe3c7141b"} Sep 17 07:08:48.250425 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:48.250377 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dzr67" podStartSLOduration=7.771325828 podStartE2EDuration="40.250361176s" totalImagesPullingTime="32.479035348s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:08:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:08:48.249207791 +0000 UTC m=+40.741282586" watchObservedRunningTime="2026-09-17 07:08:48.250361176 +0000 UTC m=+40.742435969" Sep 17 07:08:48.428695 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:48.428662 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:48.588043 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:48.588008 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:48.685733 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:48.685702 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:49.130009 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:49.129976 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:49.130220 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:49.130125 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:08:49.130220 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:49.130186 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls podName:a647a2aa-ba1d-4809-9639-dddcba088b0e nodeName:}" failed. No retries permitted until 2026-09-17 07:08:57.130170252 +0000 UTC m=+49.622245024 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls") pod "dns-default-79g8v" (UID: "a647a2aa-ba1d-4809-9639-dddcba088b0e") : secret "dns-default-metrics-tls" not found Sep 17 07:08:51.279185 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:51.279147 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:51.380111 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:51.380073 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:52.216507 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:52.216475 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:52.302490 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:52.302458 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:08:57.185780 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:08:57.185743 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:08:57.186207 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:57.185894 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:08:57.186207 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:08:57.185974 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls podName:a647a2aa-ba1d-4809-9639-dddcba088b0e nodeName:}" failed. No retries permitted until 2026-09-17 07:09:13.185957167 +0000 UTC m=+65.678031941 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls") pod "dns-default-79g8v" (UID: "a647a2aa-ba1d-4809-9639-dddcba088b0e") : secret "dns-default-metrics-tls" not found Sep 17 07:09:00.716245 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:00.716209 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:00.879476 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:00.879451 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:05.202346 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.202313 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z"] Sep 17 07:09:05.233955 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.233927 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z"] Sep 17 07:09:05.234094 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.234036 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z" Sep 17 07:09:05.237431 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.237407 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 17 07:09:05.237431 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.237417 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 17 07:09:05.237641 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.237437 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 17 07:09:05.237641 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.237447 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 17 07:09:05.237641 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.237534 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-h6dt9\"" Sep 17 07:09:05.310890 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.310853 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-9n87p"] Sep 17 07:09:05.334932 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.334891 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:05.335470 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.335442 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-9n87p"] Sep 17 07:09:05.337164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.337140 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 17 07:09:05.337397 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.337380 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-6bvkm\"" Sep 17 07:09:05.337604 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.337581 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 17 07:09:05.337968 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.337953 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 17 07:09:05.340906 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.340883 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/598a71c4-d028-4460-af48-b76302639301-config\") pod \"service-ca-operator-cf7c776b7-txr6z\" (UID: \"598a71c4-d028-4460-af48-b76302639301\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z" Sep 17 07:09:05.341000 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.340948 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/598a71c4-d028-4460-af48-b76302639301-serving-cert\") pod \"service-ca-operator-cf7c776b7-txr6z\" (UID: \"598a71c4-d028-4460-af48-b76302639301\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z" Sep 17 07:09:05.341000 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.340966 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cf9x\" (UniqueName: \"kubernetes.io/projected/598a71c4-d028-4460-af48-b76302639301-kube-api-access-7cf9x\") pod \"service-ca-operator-cf7c776b7-txr6z\" (UID: \"598a71c4-d028-4460-af48-b76302639301\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z" Sep 17 07:09:05.341930 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.341766 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 17 07:09:05.342609 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.342589 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 17 07:09:05.417115 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.417085 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-789fdfb4f-bhdgd"] Sep 17 07:09:05.441921 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.441881 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/598a71c4-d028-4460-af48-b76302639301-serving-cert\") pod \"service-ca-operator-cf7c776b7-txr6z\" (UID: \"598a71c4-d028-4460-af48-b76302639301\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z" Sep 17 07:09:05.442086 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.441924 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cf9x\" (UniqueName: \"kubernetes.io/projected/598a71c4-d028-4460-af48-b76302639301-kube-api-access-7cf9x\") pod \"service-ca-operator-cf7c776b7-txr6z\" (UID: \"598a71c4-d028-4460-af48-b76302639301\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z" Sep 17 07:09:05.442086 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.441971 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb2h6\" (UniqueName: \"kubernetes.io/projected/af922fc6-a4ff-40a5-a5d7-d86698d0a1fb-kube-api-access-jb2h6\") pod \"console-operator-696884fb9c-9n87p\" (UID: \"af922fc6-a4ff-40a5-a5d7-d86698d0a1fb\") " pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:05.442086 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.442007 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/598a71c4-d028-4460-af48-b76302639301-config\") pod \"service-ca-operator-cf7c776b7-txr6z\" (UID: \"598a71c4-d028-4460-af48-b76302639301\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z" Sep 17 07:09:05.442086 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.442030 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af922fc6-a4ff-40a5-a5d7-d86698d0a1fb-trusted-ca\") pod \"console-operator-696884fb9c-9n87p\" (UID: \"af922fc6-a4ff-40a5-a5d7-d86698d0a1fb\") " pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:05.442086 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.442052 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af922fc6-a4ff-40a5-a5d7-d86698d0a1fb-config\") pod \"console-operator-696884fb9c-9n87p\" (UID: \"af922fc6-a4ff-40a5-a5d7-d86698d0a1fb\") " pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:05.442086 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.442072 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af922fc6-a4ff-40a5-a5d7-d86698d0a1fb-serving-cert\") pod \"console-operator-696884fb9c-9n87p\" (UID: \"af922fc6-a4ff-40a5-a5d7-d86698d0a1fb\") " pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:05.442614 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.442593 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/598a71c4-d028-4460-af48-b76302639301-config\") pod \"service-ca-operator-cf7c776b7-txr6z\" (UID: \"598a71c4-d028-4460-af48-b76302639301\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z" Sep 17 07:09:05.443430 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.443404 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.444294 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.444270 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-789fdfb4f-bhdgd"] Sep 17 07:09:05.445405 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.445383 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 17 07:09:05.445489 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.445388 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/598a71c4-d028-4460-af48-b76302639301-serving-cert\") pod \"service-ca-operator-cf7c776b7-txr6z\" (UID: \"598a71c4-d028-4460-af48-b76302639301\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z" Sep 17 07:09:05.447957 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.447933 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 17 07:09:05.448544 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.448519 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 17 07:09:05.449956 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.449932 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rps6t\"" Sep 17 07:09:05.453029 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.452976 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cf9x\" (UniqueName: \"kubernetes.io/projected/598a71c4-d028-4460-af48-b76302639301-kube-api-access-7cf9x\") pod \"service-ca-operator-cf7c776b7-txr6z\" (UID: \"598a71c4-d028-4460-af48-b76302639301\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z" Sep 17 07:09:05.456317 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.455611 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 17 07:09:05.491699 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.491666 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:05.542682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.542649 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z" Sep 17 07:09:05.542833 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.542811 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb2h6\" (UniqueName: \"kubernetes.io/projected/af922fc6-a4ff-40a5-a5d7-d86698d0a1fb-kube-api-access-jb2h6\") pod \"console-operator-696884fb9c-9n87p\" (UID: \"af922fc6-a4ff-40a5-a5d7-d86698d0a1fb\") " pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:05.542889 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.542857 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af922fc6-a4ff-40a5-a5d7-d86698d0a1fb-trusted-ca\") pod \"console-operator-696884fb9c-9n87p\" (UID: \"af922fc6-a4ff-40a5-a5d7-d86698d0a1fb\") " pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:05.542889 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.542879 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/188de529-94f1-4a15-900e-fade80cfa037-installation-pull-secrets\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.542997 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.542981 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-bound-sa-token\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.543052 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.543009 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af922fc6-a4ff-40a5-a5d7-d86698d0a1fb-config\") pod \"console-operator-696884fb9c-9n87p\" (UID: \"af922fc6-a4ff-40a5-a5d7-d86698d0a1fb\") " pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:05.543052 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.543024 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af922fc6-a4ff-40a5-a5d7-d86698d0a1fb-serving-cert\") pod \"console-operator-696884fb9c-9n87p\" (UID: \"af922fc6-a4ff-40a5-a5d7-d86698d0a1fb\") " pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:05.543052 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.543042 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/188de529-94f1-4a15-900e-fade80cfa037-registry-certificates\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.543173 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.543068 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/188de529-94f1-4a15-900e-fade80cfa037-ca-trust-extracted\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.543173 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.543110 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94dkb\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-kube-api-access-94dkb\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.543264 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.543199 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.543306 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.543259 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/188de529-94f1-4a15-900e-fade80cfa037-image-registry-private-configuration\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.543343 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.543300 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/188de529-94f1-4a15-900e-fade80cfa037-trusted-ca\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.543626 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.543606 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af922fc6-a4ff-40a5-a5d7-d86698d0a1fb-config\") pod \"console-operator-696884fb9c-9n87p\" (UID: \"af922fc6-a4ff-40a5-a5d7-d86698d0a1fb\") " pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:05.543721 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.543703 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af922fc6-a4ff-40a5-a5d7-d86698d0a1fb-trusted-ca\") pod \"console-operator-696884fb9c-9n87p\" (UID: \"af922fc6-a4ff-40a5-a5d7-d86698d0a1fb\") " pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:05.545909 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.545581 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af922fc6-a4ff-40a5-a5d7-d86698d0a1fb-serving-cert\") pod \"console-operator-696884fb9c-9n87p\" (UID: \"af922fc6-a4ff-40a5-a5d7-d86698d0a1fb\") " pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:05.551585 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.551556 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb2h6\" (UniqueName: \"kubernetes.io/projected/af922fc6-a4ff-40a5-a5d7-d86698d0a1fb-kube-api-access-jb2h6\") pod \"console-operator-696884fb9c-9n87p\" (UID: \"af922fc6-a4ff-40a5-a5d7-d86698d0a1fb\") " pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:05.564949 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.564922 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:05.644133 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.644095 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:05.644299 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.644203 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/188de529-94f1-4a15-900e-fade80cfa037-image-registry-private-configuration\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.644299 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.644232 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/188de529-94f1-4a15-900e-fade80cfa037-trusted-ca\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.644299 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.644270 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/188de529-94f1-4a15-900e-fade80cfa037-installation-pull-secrets\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.644299 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.644295 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-bound-sa-token\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.644437 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.644321 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/188de529-94f1-4a15-900e-fade80cfa037-registry-certificates\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.644437 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.644350 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/188de529-94f1-4a15-900e-fade80cfa037-ca-trust-extracted\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.644560 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.644533 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94dkb\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-kube-api-access-94dkb\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.644617 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.644599 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.644756 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:05.644739 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:09:05.644861 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:05.644759 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-789fdfb4f-bhdgd: secret "image-registry-tls" not found Sep 17 07:09:05.644861 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.644818 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/188de529-94f1-4a15-900e-fade80cfa037-ca-trust-extracted\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.644861 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:05.644846 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls podName:188de529-94f1-4a15-900e-fade80cfa037 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:06.144826411 +0000 UTC m=+58.636901196 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls") pod "image-registry-789fdfb4f-bhdgd" (UID: "188de529-94f1-4a15-900e-fade80cfa037") : secret "image-registry-tls" not found Sep 17 07:09:05.645143 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.645122 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/188de529-94f1-4a15-900e-fade80cfa037-registry-certificates\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.645366 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.645351 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/188de529-94f1-4a15-900e-fade80cfa037-trusted-ca\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.646736 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.646711 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/188de529-94f1-4a15-900e-fade80cfa037-installation-pull-secrets\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.646927 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.646907 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/188de529-94f1-4a15-900e-fade80cfa037-image-registry-private-configuration\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.656970 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.656933 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94dkb\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-kube-api-access-94dkb\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.658144 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.658124 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-bound-sa-token\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:05.814634 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.814600 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z"] Sep 17 07:09:05.818356 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:09:05.818329 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod598a71c4_d028_4460_af48_b76302639301.slice/crio-33f659f9ac67d1c18228de5a99617704ebe44bcca11f6f70924904cdecebc7ff WatchSource:0}: Error finding container 33f659f9ac67d1c18228de5a99617704ebe44bcca11f6f70924904cdecebc7ff: Status 404 returned error can't find the container with id 33f659f9ac67d1c18228de5a99617704ebe44bcca11f6f70924904cdecebc7ff Sep 17 07:09:05.935633 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:05.935310 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-9n87p"] Sep 17 07:09:05.939693 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:09:05.939655 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf922fc6_a4ff_40a5_a5d7_d86698d0a1fb.slice/crio-9174a60979bd39bd5b27f6d3655a5ac4c2fa510643b81deb4353eb64b8572874 WatchSource:0}: Error finding container 9174a60979bd39bd5b27f6d3655a5ac4c2fa510643b81deb4353eb64b8572874: Status 404 returned error can't find the container with id 9174a60979bd39bd5b27f6d3655a5ac4c2fa510643b81deb4353eb64b8572874 Sep 17 07:09:06.150211 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:06.150128 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:06.150350 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:06.150247 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:09:06.150350 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:06.150258 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-789fdfb4f-bhdgd: secret "image-registry-tls" not found Sep 17 07:09:06.150350 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:06.150306 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls podName:188de529-94f1-4a15-900e-fade80cfa037 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:07.150291714 +0000 UTC m=+59.642366485 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls") pod "image-registry-789fdfb4f-bhdgd" (UID: "188de529-94f1-4a15-900e-fade80cfa037") : secret "image-registry-tls" not found Sep 17 07:09:06.255676 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:06.255640 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z" event={"ID":"598a71c4-d028-4460-af48-b76302639301","Type":"ContainerStarted","Data":"33f659f9ac67d1c18228de5a99617704ebe44bcca11f6f70924904cdecebc7ff"} Sep 17 07:09:06.256689 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:06.256660 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-9n87p" event={"ID":"af922fc6-a4ff-40a5-a5d7-d86698d0a1fb","Type":"ContainerStarted","Data":"9174a60979bd39bd5b27f6d3655a5ac4c2fa510643b81deb4353eb64b8572874"} Sep 17 07:09:07.160432 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:07.160389 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:07.160623 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:07.160558 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:09:07.160623 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:07.160581 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-789fdfb4f-bhdgd: secret "image-registry-tls" not found Sep 17 07:09:07.160732 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:07.160653 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls podName:188de529-94f1-4a15-900e-fade80cfa037 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:09.160629318 +0000 UTC m=+61.652704091 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls") pod "image-registry-789fdfb4f-bhdgd" (UID: "188de529-94f1-4a15-900e-fade80cfa037") : secret "image-registry-tls" not found Sep 17 07:09:07.371335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:07.371287 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:07.491537 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:07.491445 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:08.186059 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:08.186027 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-64lxj" Sep 17 07:09:08.489184 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:08.489155 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:08.753033 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:08.752951 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:09.176596 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.176556 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:09.176757 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:09.176718 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:09:09.176757 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:09.176737 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-789fdfb4f-bhdgd: secret "image-registry-tls" not found Sep 17 07:09:09.176872 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:09.176818 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls podName:188de529-94f1-4a15-900e-fade80cfa037 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:13.176801731 +0000 UTC m=+65.668876520 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls") pod "image-registry-789fdfb4f-bhdgd" (UID: "188de529-94f1-4a15-900e-fade80cfa037") : secret "image-registry-tls" not found Sep 17 07:09:09.266496 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.266469 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/0.log" Sep 17 07:09:09.266652 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.266505 2566 generic.go:356] "Generic (PLEG): container finished" podID="af922fc6-a4ff-40a5-a5d7-d86698d0a1fb" containerID="cf8986bc8faca884e06a610ae645ac824344aae521e34f493da8ce63973717b4" exitCode=255 Sep 17 07:09:09.266652 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.266577 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-9n87p" event={"ID":"af922fc6-a4ff-40a5-a5d7-d86698d0a1fb","Type":"ContainerDied","Data":"cf8986bc8faca884e06a610ae645ac824344aae521e34f493da8ce63973717b4"} Sep 17 07:09:09.266835 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.266816 2566 scope.go:117] "RemoveContainer" containerID="cf8986bc8faca884e06a610ae645ac824344aae521e34f493da8ce63973717b4" Sep 17 07:09:09.268222 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.268193 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z" event={"ID":"598a71c4-d028-4460-af48-b76302639301","Type":"ContainerStarted","Data":"6e8fe350ab1c3fb14726157d236ea47eeb421744b4dc8155b9a2982f604e5d92"} Sep 17 07:09:09.354968 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.354911 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z" podStartSLOduration=1.7541022179999999 podStartE2EDuration="4.354894981s" totalImagesPullingTime="2.600792763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:09:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:09:09.354491598 +0000 UTC m=+61.846566391" watchObservedRunningTime="2026-09-17 07:09:09.354894981 +0000 UTC m=+61.846969774" Sep 17 07:09:09.495431 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.495338 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:09.727445 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.727411 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:09.768926 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.768842 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-j8x88"] Sep 17 07:09:09.771885 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.771861 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:09.790259 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.790234 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 17 07:09:09.793851 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.793825 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 17 07:09:09.793851 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.793839 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hbl8z\"" Sep 17 07:09:09.801182 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.801162 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 17 07:09:09.809971 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.809953 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 17 07:09:09.811724 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.811704 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j8x88"] Sep 17 07:09:09.882516 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.882480 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:09.882516 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.882514 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:09.882712 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.882544 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-crio-socket\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:09.882712 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.882578 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjr5h\" (UniqueName: \"kubernetes.io/projected/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-api-access-wjr5h\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:09.882712 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.882615 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-data-volume\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:09.983532 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.983493 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:09.983532 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.983531 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:09.983725 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.983558 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-crio-socket\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:09.983725 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.983585 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjr5h\" (UniqueName: \"kubernetes.io/projected/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-api-access-wjr5h\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:09.983725 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:09.983612 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:09:09.983725 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:09.983664 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:09:09.983725 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:09.983684 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm podName:a9256b15-bf3a-4cad-8894-6ca06e0b7fe2 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:10.483666402 +0000 UTC m=+62.975741179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j8x88" (UID: "a9256b15-bf3a-4cad-8894-6ca06e0b7fe2") : configmap "kube-rbac-proxy" not found Sep 17 07:09:09.983725 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.983702 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-data-volume\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:09.983725 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:09.983723 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls podName:a9256b15-bf3a-4cad-8894-6ca06e0b7fe2 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:10.483708886 +0000 UTC m=+62.975783657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j8x88" (UID: "a9256b15-bf3a-4cad-8894-6ca06e0b7fe2") : secret "insights-runtime-extractor-tls" not found Sep 17 07:09:09.983989 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.983663 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-crio-socket\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:09.984033 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:09.984018 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-data-volume\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:10.015083 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:10.015047 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:10.016479 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:10.016454 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjr5h\" (UniqueName: \"kubernetes.io/projected/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-api-access-wjr5h\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:10.233807 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:10.233767 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:10.271854 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:10.271820 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/1.log" Sep 17 07:09:10.272250 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:10.272233 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/0.log" Sep 17 07:09:10.272301 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:10.272268 2566 generic.go:356] "Generic (PLEG): container finished" podID="af922fc6-a4ff-40a5-a5d7-d86698d0a1fb" containerID="52e6f8bbde59f78c7e2dbd4e88da929d5c4bdaede641a6a283c2c31f1b2b53a6" exitCode=255 Sep 17 07:09:10.272335 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:10.272317 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-9n87p" event={"ID":"af922fc6-a4ff-40a5-a5d7-d86698d0a1fb","Type":"ContainerDied","Data":"52e6f8bbde59f78c7e2dbd4e88da929d5c4bdaede641a6a283c2c31f1b2b53a6"} Sep 17 07:09:10.272387 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:10.272374 2566 scope.go:117] "RemoveContainer" containerID="cf8986bc8faca884e06a610ae645ac824344aae521e34f493da8ce63973717b4" Sep 17 07:09:10.272557 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:10.272538 2566 scope.go:117] "RemoveContainer" containerID="52e6f8bbde59f78c7e2dbd4e88da929d5c4bdaede641a6a283c2c31f1b2b53a6" Sep 17 07:09:10.272739 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:10.272720 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-9n87p_openshift-console-operator(af922fc6-a4ff-40a5-a5d7-d86698d0a1fb)\"" pod="openshift-console-operator/console-operator-696884fb9c-9n87p" podUID="af922fc6-a4ff-40a5-a5d7-d86698d0a1fb" Sep 17 07:09:10.488138 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:10.488036 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:10.488138 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:10.488088 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:10.488325 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:10.488167 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:09:10.488325 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:10.488179 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:09:10.488325 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:10.488243 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm podName:a9256b15-bf3a-4cad-8894-6ca06e0b7fe2 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:11.488224026 +0000 UTC m=+63.980298797 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j8x88" (UID: "a9256b15-bf3a-4cad-8894-6ca06e0b7fe2") : configmap "kube-rbac-proxy" not found Sep 17 07:09:10.488325 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:10.488278 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls podName:a9256b15-bf3a-4cad-8894-6ca06e0b7fe2 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:11.488266353 +0000 UTC m=+63.980341130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j8x88" (UID: "a9256b15-bf3a-4cad-8894-6ca06e0b7fe2") : secret "insights-runtime-extractor-tls" not found Sep 17 07:09:10.820803 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:10.820745 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-t7hp8"] Sep 17 07:09:10.826939 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:10.826913 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-t7hp8" Sep 17 07:09:10.834425 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:10.834397 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-vgqpb\"" Sep 17 07:09:10.885643 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:10.885615 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-t7hp8"] Sep 17 07:09:10.993285 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:10.993247 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkt68\" (UniqueName: \"kubernetes.io/projected/843c9d67-6674-4385-9d84-bdad650d593c-kube-api-access-gkt68\") pod \"network-check-source-6f8b768455-t7hp8\" (UID: \"843c9d67-6674-4385-9d84-bdad650d593c\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-t7hp8" Sep 17 07:09:11.094542 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:11.094447 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkt68\" (UniqueName: \"kubernetes.io/projected/843c9d67-6674-4385-9d84-bdad650d593c-kube-api-access-gkt68\") pod \"network-check-source-6f8b768455-t7hp8\" (UID: \"843c9d67-6674-4385-9d84-bdad650d593c\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-t7hp8" Sep 17 07:09:11.107104 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:11.107079 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 07:09:11.123524 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:11.123492 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 07:09:11.127863 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:11.127836 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkt68\" (UniqueName: \"kubernetes.io/projected/843c9d67-6674-4385-9d84-bdad650d593c-kube-api-access-gkt68\") pod \"network-check-source-6f8b768455-t7hp8\" (UID: \"843c9d67-6674-4385-9d84-bdad650d593c\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-t7hp8" Sep 17 07:09:11.137651 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:11.137631 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-t7hp8" Sep 17 07:09:11.276428 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:11.276400 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/1.log" Sep 17 07:09:11.276736 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:11.276721 2566 scope.go:117] "RemoveContainer" containerID="52e6f8bbde59f78c7e2dbd4e88da929d5c4bdaede641a6a283c2c31f1b2b53a6" Sep 17 07:09:11.276912 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:11.276891 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-9n87p_openshift-console-operator(af922fc6-a4ff-40a5-a5d7-d86698d0a1fb)\"" pod="openshift-console-operator/console-operator-696884fb9c-9n87p" podUID="af922fc6-a4ff-40a5-a5d7-d86698d0a1fb" Sep 17 07:09:11.420519 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:11.420431 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-t7hp8"] Sep 17 07:09:11.424655 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:09:11.424618 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod843c9d67_6674_4385_9d84_bdad650d593c.slice/crio-a3033af290fa57a693ba363c2923d2dfd23d4fe18b7bb80d9c7e3457402aa550 WatchSource:0}: Error finding container a3033af290fa57a693ba363c2923d2dfd23d4fe18b7bb80d9c7e3457402aa550: Status 404 returned error can't find the container with id a3033af290fa57a693ba363c2923d2dfd23d4fe18b7bb80d9c7e3457402aa550 Sep 17 07:09:11.498162 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:11.498121 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:11.498162 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:11.498165 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:11.498418 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:11.498266 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:09:11.498418 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:11.498269 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:09:11.498418 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:11.498339 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm podName:a9256b15-bf3a-4cad-8894-6ca06e0b7fe2 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:13.49832134 +0000 UTC m=+65.990396110 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j8x88" (UID: "a9256b15-bf3a-4cad-8894-6ca06e0b7fe2") : configmap "kube-rbac-proxy" not found Sep 17 07:09:11.498418 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:11.498363 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls podName:a9256b15-bf3a-4cad-8894-6ca06e0b7fe2 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:13.498350838 +0000 UTC m=+65.990425614 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j8x88" (UID: "a9256b15-bf3a-4cad-8894-6ca06e0b7fe2") : secret "insights-runtime-extractor-tls" not found Sep 17 07:09:11.819585 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:11.819560 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wffc2_e94051ac-fe49-4338-ab5f-69979421a9cb/dns-node-resolver/0.log" Sep 17 07:09:11.859722 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:11.859689 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:11.969982 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:11.969952 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:12.195292 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:12.195202 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:12.280518 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:12.280484 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-t7hp8" event={"ID":"843c9d67-6674-4385-9d84-bdad650d593c","Type":"ContainerStarted","Data":"a3033af290fa57a693ba363c2923d2dfd23d4fe18b7bb80d9c7e3457402aa550"} Sep 17 07:09:12.450173 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:12.450095 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:12.804506 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:12.804475 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kvftr_7a9610b5-26e5-44c9-a253-ea18eaec24af/node-ca/0.log" Sep 17 07:09:12.810124 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:12.810090 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs\") pod \"network-metrics-daemon-z2l2s\" (UID: \"dc4c353d-0b67-4b66-a052-dd8397466d6f\") " pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:09:12.810267 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:12.810206 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret\") pod \"global-pull-secret-syncer-7jt2z\" (UID: \"e59c89a5-bae3-48e6-bdb5-f52094590f45\") " pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:09:12.813899 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:12.813871 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 07:09:12.814020 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:12.813909 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 07:09:12.821050 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:12.821017 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 17 07:09:12.821162 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:12.821087 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs podName:dc4c353d-0b67-4b66-a052-dd8397466d6f nodeName:}" failed. No retries permitted until 2026-09-17 07:10:16.821066709 +0000 UTC m=+129.313141494 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs") pod "network-metrics-daemon-z2l2s" (UID: "dc4c353d-0b67-4b66-a052-dd8397466d6f") : secret "metrics-daemon-secret" not found Sep 17 07:09:12.823536 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:12.823509 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e59c89a5-bae3-48e6-bdb5-f52094590f45-original-pull-secret\") pod \"global-pull-secret-syncer-7jt2z\" (UID: \"e59c89a5-bae3-48e6-bdb5-f52094590f45\") " pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:09:12.910944 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:12.910907 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg4sk\" (UniqueName: \"kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk\") pod \"network-check-target-vddmm\" (UID: \"3b975ace-eff0-4687-845a-c9607f77ac13\") " pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:09:12.913965 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:12.913941 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg4sk\" (UniqueName: \"kubernetes.io/projected/3b975ace-eff0-4687-845a-c9607f77ac13-kube-api-access-hg4sk\") pod \"network-check-target-vddmm\" (UID: \"3b975ace-eff0-4687-845a-c9607f77ac13\") " pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:09:12.940229 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:12.940196 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2f8p7\"" Sep 17 07:09:12.942291 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:12.942274 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7jt2z" Sep 17 07:09:12.948261 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:12.948235 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:09:13.199317 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:13.199243 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2n76n_f70632fa-c15c-4beb-8c27-2b4316baafd5/serve-healthcheck-canary/0.log" Sep 17 07:09:13.213570 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:13.213537 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:09:13.213735 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:13.213585 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:13.213735 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:13.213682 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:09:13.213735 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:13.213702 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:09:13.213735 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:13.213717 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-789fdfb4f-bhdgd: secret "image-registry-tls" not found Sep 17 07:09:13.213990 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:13.213756 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls podName:a647a2aa-ba1d-4809-9639-dddcba088b0e nodeName:}" failed. No retries permitted until 2026-09-17 07:09:45.213735499 +0000 UTC m=+97.705810270 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls") pod "dns-default-79g8v" (UID: "a647a2aa-ba1d-4809-9639-dddcba088b0e") : secret "dns-default-metrics-tls" not found Sep 17 07:09:13.213990 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:13.213776 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls podName:188de529-94f1-4a15-900e-fade80cfa037 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:21.213766082 +0000 UTC m=+73.705840868 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls") pod "image-registry-789fdfb4f-bhdgd" (UID: "188de529-94f1-4a15-900e-fade80cfa037") : secret "image-registry-tls" not found Sep 17 07:09:13.250132 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:13.250100 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7jt2z"] Sep 17 07:09:13.256269 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:09:13.256243 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode59c89a5_bae3_48e6_bdb5_f52094590f45.slice/crio-337ada364ffb5cf99a2858fc9a882a662d93d7289a38dff38cab236bf740cb0d WatchSource:0}: Error finding container 337ada364ffb5cf99a2858fc9a882a662d93d7289a38dff38cab236bf740cb0d: Status 404 returned error can't find the container with id 337ada364ffb5cf99a2858fc9a882a662d93d7289a38dff38cab236bf740cb0d Sep 17 07:09:13.272046 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:13.272014 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vddmm"] Sep 17 07:09:13.275537 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:09:13.275512 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b975ace_eff0_4687_845a_c9607f77ac13.slice/crio-7439af0da8cf2b039b408a0f955f1875e49ddd5199c7ea221d02b7d997211a1f WatchSource:0}: Error finding container 7439af0da8cf2b039b408a0f955f1875e49ddd5199c7ea221d02b7d997211a1f: Status 404 returned error can't find the container with id 7439af0da8cf2b039b408a0f955f1875e49ddd5199c7ea221d02b7d997211a1f Sep 17 07:09:13.283764 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:13.283738 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vddmm" event={"ID":"3b975ace-eff0-4687-845a-c9607f77ac13","Type":"ContainerStarted","Data":"7439af0da8cf2b039b408a0f955f1875e49ddd5199c7ea221d02b7d997211a1f"} Sep 17 07:09:13.284684 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:13.284665 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7jt2z" event={"ID":"e59c89a5-bae3-48e6-bdb5-f52094590f45","Type":"ContainerStarted","Data":"337ada364ffb5cf99a2858fc9a882a662d93d7289a38dff38cab236bf740cb0d"} Sep 17 07:09:13.515384 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:13.515295 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:13.515384 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:13.515332 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:13.515608 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:13.515439 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:09:13.515608 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:13.515518 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm podName:a9256b15-bf3a-4cad-8894-6ca06e0b7fe2 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:17.51549668 +0000 UTC m=+70.007571475 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j8x88" (UID: "a9256b15-bf3a-4cad-8894-6ca06e0b7fe2") : configmap "kube-rbac-proxy" not found Sep 17 07:09:13.515608 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:13.515520 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:09:13.515608 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:13.515573 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls podName:a9256b15-bf3a-4cad-8894-6ca06e0b7fe2 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:17.515557521 +0000 UTC m=+70.007632299 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j8x88" (UID: "a9256b15-bf3a-4cad-8894-6ca06e0b7fe2") : secret "insights-runtime-extractor-tls" not found Sep 17 07:09:14.385347 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:14.385312 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:15.644847 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:15.644808 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:15.645269 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:15.644877 2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:15.645349 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:15.645297 2566 scope.go:117] "RemoveContainer" containerID="52e6f8bbde59f78c7e2dbd4e88da929d5c4bdaede641a6a283c2c31f1b2b53a6" Sep 17 07:09:15.645525 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:15.645499 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-9n87p_openshift-console-operator(af922fc6-a4ff-40a5-a5d7-d86698d0a1fb)\"" pod="openshift-console-operator/console-operator-696884fb9c-9n87p" podUID="af922fc6-a4ff-40a5-a5d7-d86698d0a1fb" Sep 17 07:09:15.838513 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:15.838481 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:16.094669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:16.094630 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:16.297958 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:16.297913 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vddmm" event={"ID":"3b975ace-eff0-4687-845a-c9607f77ac13","Type":"ContainerStarted","Data":"02cdc4d2cf06984d335cc0e884c170026b7a6ac530250439b593ed34d0f8c935"} Sep 17 07:09:16.298148 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:16.298010 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:09:16.299448 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:16.299420 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-t7hp8" event={"ID":"843c9d67-6674-4385-9d84-bdad650d593c","Type":"ContainerStarted","Data":"4ec7205bcfbfa37aaaf5d44d96ff6a86eb84a56cd7fde85e57c8a72dfb47b1db"} Sep 17 07:09:16.310906 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:16.310878 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:16.447128 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:16.447033 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vddmm" podStartSLOduration=66.046361158 podStartE2EDuration="1m8.447014434s" totalImagesPullingTime="2.400653276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:09:16.404105413 +0000 UTC m=+68.896180209" watchObservedRunningTime="2026-09-17 07:09:16.447014434 +0000 UTC m=+68.939089228" Sep 17 07:09:16.447441 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:16.447420 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-t7hp8" podStartSLOduration=2.454947539 podStartE2EDuration="6.447412537s" totalImagesPullingTime="3.992464998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:09:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:09:16.443329739 +0000 UTC m=+68.935404533" watchObservedRunningTime="2026-09-17 07:09:16.447412537 +0000 UTC m=+68.939487329" Sep 17 07:09:16.530452 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:16.530413 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:17.316368 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:17.316329 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:17.545195 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:17.545151 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:17.545372 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:17.545206 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:17.545372 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:17.545258 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:09:17.545372 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:17.545341 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm podName:a9256b15-bf3a-4cad-8894-6ca06e0b7fe2 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:25.545318352 +0000 UTC m=+78.037393123 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j8x88" (UID: "a9256b15-bf3a-4cad-8894-6ca06e0b7fe2") : configmap "kube-rbac-proxy" not found Sep 17 07:09:17.545550 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:17.545383 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:09:17.545550 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:17.545451 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls podName:a9256b15-bf3a-4cad-8894-6ca06e0b7fe2 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:25.545432855 +0000 UTC m=+78.037507649 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j8x88" (UID: "a9256b15-bf3a-4cad-8894-6ca06e0b7fe2") : secret "insights-runtime-extractor-tls" not found Sep 17 07:09:17.721731 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:17.721648 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:18.080557 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:18.080533 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:18.283854 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:18.283822 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:18.925331 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:18.925294 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:19.310502 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:19.310467 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7jt2z" event={"ID":"e59c89a5-bae3-48e6-bdb5-f52094590f45","Type":"ContainerStarted","Data":"da3256f9113cc26ed74e78ba703bfc8d3fcf99f2e35ca14322cec1a3c3e058b0"} Sep 17 07:09:19.361469 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:19.361434 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:19.377596 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:19.377551 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7jt2z" podStartSLOduration=66.245170552 podStartE2EDuration="1m11.377531186s" totalImagesPullingTime="5.132360634s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:09:19.367733938 +0000 UTC m=+71.859808730" watchObservedRunningTime="2026-09-17 07:09:19.377531186 +0000 UTC m=+71.869605984" Sep 17 07:09:19.607448 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:19.607369 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:19.746247 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:19.746213 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:20.067321 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:20.067290 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:20.503126 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:20.503041 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:20.680417 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:20.680387 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:20.812816 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:20.812766 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:21.273001 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:21.272905 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:21.275277 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:21.275256 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls\") pod \"image-registry-789fdfb4f-bhdgd\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:21.335270 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:21.335240 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:21.365116 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:21.365093 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rps6t\"" Sep 17 07:09:21.372604 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:21.372583 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:21.529266 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:21.529178 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:21.670883 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:21.670852 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-789fdfb4f-bhdgd"] Sep 17 07:09:21.674159 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:09:21.674134 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod188de529_94f1_4a15_900e_fade80cfa037.slice/crio-e3d2668ec9913796abe2fb29e486a2f6e4418b256520969bd7bd8d0d2bb1713f WatchSource:0}: Error finding container e3d2668ec9913796abe2fb29e486a2f6e4418b256520969bd7bd8d0d2bb1713f: Status 404 returned error can't find the container with id e3d2668ec9913796abe2fb29e486a2f6e4418b256520969bd7bd8d0d2bb1713f Sep 17 07:09:21.726659 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:21.726629 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:21.981174 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:21.981144 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:22.311548 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:22.311516 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:22.321153 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:22.321129 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" event={"ID":"188de529-94f1-4a15-900e-fade80cfa037","Type":"ContainerStarted","Data":"d8aea0cc13e63858bf845f75030c62e42840daa7e10870b239d78e6896cb17fc"} Sep 17 07:09:22.321283 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:22.321174 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" event={"ID":"188de529-94f1-4a15-900e-fade80cfa037","Type":"ContainerStarted","Data":"e3d2668ec9913796abe2fb29e486a2f6e4418b256520969bd7bd8d0d2bb1713f"} Sep 17 07:09:22.321283 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:22.321257 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:22.368429 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:22.368380 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" podStartSLOduration=17.368364216 podStartE2EDuration="17.368364216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:09:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:09:22.368295899 +0000 UTC m=+74.860370705" watchObservedRunningTime="2026-09-17 07:09:22.368364216 +0000 UTC m=+74.860439009" Sep 17 07:09:22.684997 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:22.684916 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:22.854486 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:22.854453 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:23.462581 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:23.462547 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:23.645973 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:23.645932 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:23.851991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:23.851962 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:24.433234 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:24.433200 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:24.952103 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:24.952075 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:25.254456 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:25.254371 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:25.601458 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:25.601428 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:25.601458 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:25.601458 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:25.601652 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:25.601586 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:09:25.601652 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:25.601651 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls podName:a9256b15-bf3a-4cad-8894-6ca06e0b7fe2 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:41.60163562 +0000 UTC m=+94.093710393 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j8x88" (UID: "a9256b15-bf3a-4cad-8894-6ca06e0b7fe2") : secret "insights-runtime-extractor-tls" not found Sep 17 07:09:25.601990 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:25.601970 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:25.681922 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:25.681880 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:25.815242 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:25.815205 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:25.959565 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:25.959487 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:26.240431 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:26.240352 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:26.416379 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:26.416344 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:26.584241 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:26.584210 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:31.018823 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:31.018673 2566 scope.go:117] "RemoveContainer" containerID="52e6f8bbde59f78c7e2dbd4e88da929d5c4bdaede641a6a283c2c31f1b2b53a6" Sep 17 07:09:31.351963 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:31.351939 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/1.log" Sep 17 07:09:31.352209 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:31.352004 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-9n87p" event={"ID":"af922fc6-a4ff-40a5-a5d7-d86698d0a1fb","Type":"ContainerStarted","Data":"b6e102a1ff16a4ed47c8aa1b9c823949fe2572581d2caf5babee03dd7f28e72e"} Sep 17 07:09:31.352294 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:31.352275 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:31.369697 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:31.369650 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-9n87p" podStartSLOduration=23.893408023 podStartE2EDuration="26.369636108s" totalImagesPullingTime="2.476228085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:09:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:09:31.369428723 +0000 UTC m=+83.861503516" watchObservedRunningTime="2026-09-17 07:09:31.369636108 +0000 UTC m=+83.861710901" Sep 17 07:09:32.063890 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:32.063861 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-9n87p" Sep 17 07:09:32.250508 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:32.250475 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:32.351474 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:32.351393 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:32.424488 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:32.424455 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:32.771985 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:32.771903 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:33.037728 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:33.037646 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:33.193071 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:33.193040 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:33.346355 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:33.346324 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:33.788409 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:33.788336 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:33.974080 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:33.974050 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:34.255555 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:34.255521 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:34.361558 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:34.361531 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:34.794516 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:34.794484 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:34.899339 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:34.899308 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:38.953165 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:38.953134 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-789fdfb4f-bhdgd"] Sep 17 07:09:38.957585 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:38.957555 2566 patch_prober.go:28] interesting pod/image-registry-789fdfb4f-bhdgd container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 17 07:09:38.957714 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:38.957615 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" podUID="188de529-94f1-4a15-900e-fade80cfa037" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 07:09:39.013138 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.013105 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d5f59f4d-bldk5"] Sep 17 07:09:39.016220 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.016204 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d5f59f4d-bldk5" Sep 17 07:09:39.019948 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.019928 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 17 07:09:39.025003 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.024983 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8"] Sep 17 07:09:39.027516 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.027494 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 17 07:09:39.027835 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.027821 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.028658 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.028641 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-gjqmh\"" Sep 17 07:09:39.028810 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.028774 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 17 07:09:39.031556 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.031541 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 17 07:09:39.040644 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.040618 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 17 07:09:39.040769 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.040745 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 17 07:09:39.040893 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.040859 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 17 07:09:39.041333 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.041313 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 17 07:09:39.096299 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.096257 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n9tx\" (UniqueName: \"kubernetes.io/projected/c3e96237-c4af-4deb-8899-970f998067b7-kube-api-access-2n9tx\") pod \"managed-serviceaccount-addon-agent-d5f59f4d-bldk5\" (UID: \"c3e96237-c4af-4deb-8899-970f998067b7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d5f59f4d-bldk5" Sep 17 07:09:39.096299 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.096301 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/70f008e5-bd5a-444a-bc32-f5a5a405895d-hub\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.096525 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.096343 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/70f008e5-bd5a-444a-bc32-f5a5a405895d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.096525 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.096361 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/70f008e5-bd5a-444a-bc32-f5a5a405895d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.096525 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.096455 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fskg\" (UniqueName: \"kubernetes.io/projected/70f008e5-bd5a-444a-bc32-f5a5a405895d-kube-api-access-9fskg\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.096525 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.096483 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3e96237-c4af-4deb-8899-970f998067b7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-d5f59f4d-bldk5\" (UID: \"c3e96237-c4af-4deb-8899-970f998067b7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d5f59f4d-bldk5" Sep 17 07:09:39.096525 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.096510 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/70f008e5-bd5a-444a-bc32-f5a5a405895d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.096676 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.096556 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/70f008e5-bd5a-444a-bc32-f5a5a405895d-ca\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.140926 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.140887 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d5f59f4d-bldk5"] Sep 17 07:09:39.141774 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.141753 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8"] Sep 17 07:09:39.149768 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.149741 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:39.197026 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.196988 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n9tx\" (UniqueName: \"kubernetes.io/projected/c3e96237-c4af-4deb-8899-970f998067b7-kube-api-access-2n9tx\") pod \"managed-serviceaccount-addon-agent-d5f59f4d-bldk5\" (UID: \"c3e96237-c4af-4deb-8899-970f998067b7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d5f59f4d-bldk5" Sep 17 07:09:39.197026 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.197026 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/70f008e5-bd5a-444a-bc32-f5a5a405895d-hub\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.197268 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.197066 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/70f008e5-bd5a-444a-bc32-f5a5a405895d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.197268 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.197084 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/70f008e5-bd5a-444a-bc32-f5a5a405895d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.197268 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.197115 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fskg\" (UniqueName: \"kubernetes.io/projected/70f008e5-bd5a-444a-bc32-f5a5a405895d-kube-api-access-9fskg\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.197268 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.197140 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3e96237-c4af-4deb-8899-970f998067b7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-d5f59f4d-bldk5\" (UID: \"c3e96237-c4af-4deb-8899-970f998067b7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d5f59f4d-bldk5" Sep 17 07:09:39.197463 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.197271 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/70f008e5-bd5a-444a-bc32-f5a5a405895d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.197463 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.197335 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/70f008e5-bd5a-444a-bc32-f5a5a405895d-ca\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.198105 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.198079 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/70f008e5-bd5a-444a-bc32-f5a5a405895d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.199874 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.199847 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/70f008e5-bd5a-444a-bc32-f5a5a405895d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.199975 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.199937 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3e96237-c4af-4deb-8899-970f998067b7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-d5f59f4d-bldk5\" (UID: \"c3e96237-c4af-4deb-8899-970f998067b7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d5f59f4d-bldk5" Sep 17 07:09:39.199975 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.199968 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/70f008e5-bd5a-444a-bc32-f5a5a405895d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.200107 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.200085 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/70f008e5-bd5a-444a-bc32-f5a5a405895d-ca\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.200174 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.200137 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/70f008e5-bd5a-444a-bc32-f5a5a405895d-hub\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.220356 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.220300 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fskg\" (UniqueName: \"kubernetes.io/projected/70f008e5-bd5a-444a-bc32-f5a5a405895d-kube-api-access-9fskg\") pod \"cluster-proxy-proxy-agent-9f5644955-fkxm8\" (UID: \"70f008e5-bd5a-444a-bc32-f5a5a405895d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.221033 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.221012 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n9tx\" (UniqueName: \"kubernetes.io/projected/c3e96237-c4af-4deb-8899-970f998067b7-kube-api-access-2n9tx\") pod \"managed-serviceaccount-addon-agent-d5f59f4d-bldk5\" (UID: \"c3e96237-c4af-4deb-8899-970f998067b7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d5f59f4d-bldk5" Sep 17 07:09:39.338526 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.338490 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:39.340361 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.340342 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d5f59f4d-bldk5" Sep 17 07:09:39.347323 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.347308 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" Sep 17 07:09:39.627817 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.627768 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d5f59f4d-bldk5"] Sep 17 07:09:39.631073 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:09:39.631049 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3e96237_c4af_4deb_8899_970f998067b7.slice/crio-b2227dbd17e6325c839c94dc650cf4e2c044f9dfc37fc00c88b5ddc52e90e070 WatchSource:0}: Error finding container b2227dbd17e6325c839c94dc650cf4e2c044f9dfc37fc00c88b5ddc52e90e070: Status 404 returned error can't find the container with id b2227dbd17e6325c839c94dc650cf4e2c044f9dfc37fc00c88b5ddc52e90e070 Sep 17 07:09:39.650418 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:39.650349 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8"] Sep 17 07:09:39.653586 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:09:39.653563 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70f008e5_bd5a_444a_bc32_f5a5a405895d.slice/crio-6162790292d8fea2966876f6071b063f3e249d445dcb65aacd2b19a7c0d88cf2 WatchSource:0}: Error finding container 6162790292d8fea2966876f6071b063f3e249d445dcb65aacd2b19a7c0d88cf2: Status 404 returned error can't find the container with id 6162790292d8fea2966876f6071b063f3e249d445dcb65aacd2b19a7c0d88cf2 Sep 17 07:09:40.273304 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:40.273271 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:40.388530 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:40.388454 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d5f59f4d-bldk5" event={"ID":"c3e96237-c4af-4deb-8899-970f998067b7","Type":"ContainerStarted","Data":"b2227dbd17e6325c839c94dc650cf4e2c044f9dfc37fc00c88b5ddc52e90e070"} Sep 17 07:09:40.390477 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:40.390138 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" event={"ID":"70f008e5-bd5a-444a-bc32-f5a5a405895d","Type":"ContainerStarted","Data":"6162790292d8fea2966876f6071b063f3e249d445dcb65aacd2b19a7c0d88cf2"} Sep 17 07:09:40.469703 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:40.469624 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:41.122620 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:41.122587 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:41.619331 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:41.619288 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:41.622334 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:41.622301 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9256b15-bf3a-4cad-8894-6ca06e0b7fe2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8x88\" (UID: \"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2\") " pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:41.879946 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:41.879866 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j8x88" Sep 17 07:09:42.423960 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:42.423922 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:42.538716 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:42.538692 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:42.705734 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:42.705597 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j8x88"] Sep 17 07:09:43.402280 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:43.402243 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d5f59f4d-bldk5" event={"ID":"c3e96237-c4af-4deb-8899-970f998067b7","Type":"ContainerStarted","Data":"f359712114315317849cbdb4d6312a61045c4adb082682d6f6b97e4277e766f5"} Sep 17 07:09:43.423772 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:43.423479 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d5f59f4d-bldk5" podStartSLOduration=2.575558348 podStartE2EDuration="5.423459422s" totalImagesPullingTime="2.847901074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:09:43.422775176 +0000 UTC m=+95.914849971" watchObservedRunningTime="2026-09-17 07:09:43.423459422 +0000 UTC m=+95.915534216" Sep 17 07:09:43.626342 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:09:43.626310 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9256b15_bf3a_4cad_8894_6ca06e0b7fe2.slice/crio-684aa4eb824c4237a1c144631279b09f4af7ba1ad36d147d8ab8b127431027f6 WatchSource:0}: Error finding container 684aa4eb824c4237a1c144631279b09f4af7ba1ad36d147d8ab8b127431027f6: Status 404 returned error can't find the container with id 684aa4eb824c4237a1c144631279b09f4af7ba1ad36d147d8ab8b127431027f6 Sep 17 07:09:44.407945 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:44.407918 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" event={"ID":"70f008e5-bd5a-444a-bc32-f5a5a405895d","Type":"ContainerStarted","Data":"51226d7b0b6a15ae2420c18c0165f0f743eda1ce5ca8fd3c0c9dfadea471db34"} Sep 17 07:09:44.408288 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:44.407963 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" event={"ID":"70f008e5-bd5a-444a-bc32-f5a5a405895d","Type":"ContainerStarted","Data":"88252f14d0d740cd3701c4b3ada173745ec30866a4068ccc5c0c13ada9ebd168"} Sep 17 07:09:44.408288 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:44.407983 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" event={"ID":"70f008e5-bd5a-444a-bc32-f5a5a405895d","Type":"ContainerStarted","Data":"6b064f7540666470db2fc931bfb9067908a578b474c409a8209c0aeae8d0a63a"} Sep 17 07:09:44.409519 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:44.409496 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j8x88" event={"ID":"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2","Type":"ContainerStarted","Data":"4065eb57f814756cdf8ac750521bb4dfeeb2e820f699788225c6af86146c3dff"} Sep 17 07:09:44.409598 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:44.409552 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j8x88" event={"ID":"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2","Type":"ContainerStarted","Data":"684aa4eb824c4237a1c144631279b09f4af7ba1ad36d147d8ab8b127431027f6"} Sep 17 07:09:44.433260 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:44.430244 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9f5644955-fkxm8" podStartSLOduration=2.391397861 podStartE2EDuration="6.430220917s" totalImagesPullingTime="4.038823056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:09:44.427958682 +0000 UTC m=+96.920033474" watchObservedRunningTime="2026-09-17 07:09:44.430220917 +0000 UTC m=+96.922295711" Sep 17 07:09:45.250870 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:45.250831 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:09:45.253403 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:45.253380 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a647a2aa-ba1d-4809-9639-dddcba088b0e-metrics-tls\") pod \"dns-default-79g8v\" (UID: \"a647a2aa-ba1d-4809-9639-dddcba088b0e\") " pod="openshift-dns/dns-default-79g8v" Sep 17 07:09:45.324881 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:45.324846 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wq5pm\"" Sep 17 07:09:45.333013 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:45.332988 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-79g8v" Sep 17 07:09:45.415801 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:45.415706 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j8x88" event={"ID":"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2","Type":"ContainerStarted","Data":"9e243730bfeebc921b4afc8e71d9f7a0f16e14cd6e8958a20feda992a669d15e"} Sep 17 07:09:45.506693 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:45.506605 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:45.581011 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:45.580975 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:45.606381 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:45.606350 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-79g8v"] Sep 17 07:09:45.728642 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:09:45.728612 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda647a2aa_ba1d_4809_9639_dddcba088b0e.slice/crio-2aa3bb11af35735f22b4f45a8d76c2aee8b2a0f835f1801c1a3c175c96768443 WatchSource:0}: Error finding container 2aa3bb11af35735f22b4f45a8d76c2aee8b2a0f835f1801c1a3c175c96768443: Status 404 returned error can't find the container with id 2aa3bb11af35735f22b4f45a8d76c2aee8b2a0f835f1801c1a3c175c96768443 Sep 17 07:09:45.937864 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:45.937830 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:46.421501 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:46.421464 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-79g8v" event={"ID":"a647a2aa-ba1d-4809-9639-dddcba088b0e","Type":"ContainerStarted","Data":"2aa3bb11af35735f22b4f45a8d76c2aee8b2a0f835f1801c1a3c175c96768443"} Sep 17 07:09:46.424084 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:46.424057 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j8x88" event={"ID":"a9256b15-bf3a-4cad-8894-6ca06e0b7fe2","Type":"ContainerStarted","Data":"d82c726e82798d0ffd1d9e9d08f34213a73a5b2948984370c675c6d83bdeeb27"} Sep 17 07:09:46.446970 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:46.446888 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-j8x88" podStartSLOduration=35.411528345 podStartE2EDuration="37.446864571s" totalImagesPullingTime="2.035336226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:09:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:09:46.445753121 +0000 UTC m=+98.937827924" watchObservedRunningTime="2026-09-17 07:09:46.446864571 +0000 UTC m=+98.938939365" Sep 17 07:09:47.305700 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:47.305670 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vddmm" Sep 17 07:09:48.363369 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:48.363284 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:48.437251 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:48.437217 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-79g8v" event={"ID":"a647a2aa-ba1d-4809-9639-dddcba088b0e","Type":"ContainerStarted","Data":"b704ec2997e5ca0b32cc8e48628e2aded5599c3c3931a6f15725ea56d0f18617"} Sep 17 07:09:48.437426 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:48.437289 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-79g8v" event={"ID":"a647a2aa-ba1d-4809-9639-dddcba088b0e","Type":"ContainerStarted","Data":"639b3f340a2333900e0077ed0d82319d522a1634e4a0dbbd83e32a807c403842"} Sep 17 07:09:48.437426 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:48.437325 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-79g8v" Sep 17 07:09:48.455774 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:48.455723 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-79g8v" podStartSLOduration=65.323456861 podStartE2EDuration="1m7.455709086s" totalImagesPullingTime="2.132252225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:09:48.454982347 +0000 UTC m=+100.947057139" watchObservedRunningTime="2026-09-17 07:09:48.455709086 +0000 UTC m=+100.947783883" Sep 17 07:09:48.462358 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:48.462333 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:48.675265 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:48.675179 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:48.958272 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:48.958192 2566 patch_prober.go:28] interesting pod/image-registry-789fdfb4f-bhdgd container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 17 07:09:48.958272 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:48.958251 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" podUID="188de529-94f1-4a15-900e-fade80cfa037" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 07:09:50.679845 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:50.679772 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:51.027326 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.027244 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6fk95"] Sep 17 07:09:51.030795 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.030771 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.041153 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.041132 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 17 07:09:51.041153 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.041142 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 17 07:09:51.041453 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.041222 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 17 07:09:51.041529 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.041435 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 17 07:09:51.041927 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.041914 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 17 07:09:51.041996 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.041944 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 17 07:09:51.041996 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.041986 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-795dp\"" Sep 17 07:09:51.099409 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.099374 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-wtmp\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.099409 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.099408 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-root\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.099670 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.099438 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-tls\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.099670 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.099545 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-textfile\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.099670 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.099575 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-sys\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.099670 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.099604 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.099670 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.099648 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-metrics-client-ca\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.099898 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.099683 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-accelerators-collector-config\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.099898 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.099714 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7tsj\" (UniqueName: \"kubernetes.io/projected/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-kube-api-access-b7tsj\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.111096 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.111067 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:51.200496 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.200462 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-textfile\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.200496 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.200499 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-sys\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.200746 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.200526 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.200746 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.200551 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-metrics-client-ca\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.200746 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.200572 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-accelerators-collector-config\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.200746 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.200592 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7tsj\" (UniqueName: \"kubernetes.io/projected/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-kube-api-access-b7tsj\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.200746 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.200607 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-sys\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.200746 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.200734 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-wtmp\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.201037 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.200780 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-root\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.201037 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.200828 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-tls\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.201037 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.200872 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-textfile\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.201037 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.200934 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-root\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.201037 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.200941 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-wtmp\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.201037 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:51.200956 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 17 07:09:51.201224 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:09:51.201071 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-tls podName:0778a52f-26ea-49d3-b3f6-8ecf5e874e85 nodeName:}" failed. No retries permitted until 2026-09-17 07:09:51.701050081 +0000 UTC m=+104.193124851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-tls") pod "node-exporter-6fk95" (UID: "0778a52f-26ea-49d3-b3f6-8ecf5e874e85") : secret "node-exporter-tls" not found Sep 17 07:09:51.201224 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.201167 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-metrics-client-ca\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.201304 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.201242 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-accelerators-collector-config\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.202872 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.202853 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.207255 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.207224 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:51.211273 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.211251 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7tsj\" (UniqueName: \"kubernetes.io/projected/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-kube-api-access-b7tsj\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.306907 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.306869 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:51.495970 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.495934 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:51.596310 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.596222 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:51.696528 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.696501 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:51.704620 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.704597 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-tls\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.706974 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.706958 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0778a52f-26ea-49d3-b3f6-8ecf5e874e85-node-exporter-tls\") pod \"node-exporter-6fk95\" (UID: \"0778a52f-26ea-49d3-b3f6-8ecf5e874e85\") " pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.939449 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.939365 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6fk95" Sep 17 07:09:51.952763 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:09:51.952735 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0778a52f_26ea_49d3_b3f6_8ecf5e874e85.slice/crio-0b112606289f7b68d13d1baa52d949f6d145c0b06967a3328c4f62ea2ddfc84c WatchSource:0}: Error finding container 0b112606289f7b68d13d1baa52d949f6d145c0b06967a3328c4f62ea2ddfc84c: Status 404 returned error can't find the container with id 0b112606289f7b68d13d1baa52d949f6d145c0b06967a3328c4f62ea2ddfc84c Sep 17 07:09:51.964993 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:51.964969 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:52.066452 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:52.066420 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:52.187098 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:52.187063 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:52.366091 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:52.366058 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:52.452164 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:52.452132 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6fk95" event={"ID":"0778a52f-26ea-49d3-b3f6-8ecf5e874e85","Type":"ContainerStarted","Data":"0b112606289f7b68d13d1baa52d949f6d145c0b06967a3328c4f62ea2ddfc84c"} Sep 17 07:09:52.542125 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:52.542091 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:53.457527 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:53.457489 2566 generic.go:356] "Generic (PLEG): container finished" podID="0778a52f-26ea-49d3-b3f6-8ecf5e874e85" containerID="c83e8579b286611c50b2239f3e09760ab683301a8e64a556dd49b536169ca576" exitCode=0 Sep 17 07:09:53.457999 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:53.457577 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6fk95" event={"ID":"0778a52f-26ea-49d3-b3f6-8ecf5e874e85","Type":"ContainerDied","Data":"c83e8579b286611c50b2239f3e09760ab683301a8e64a556dd49b536169ca576"} Sep 17 07:09:54.125830 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:54.125779 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:54.275107 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:54.275076 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:54.404369 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:54.404292 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:54.463428 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:54.463393 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6fk95" event={"ID":"0778a52f-26ea-49d3-b3f6-8ecf5e874e85","Type":"ContainerStarted","Data":"647f38abd1a764828c79b68173ffbe30821a9027088be86ebf71c431c59ad176"} Sep 17 07:09:54.463930 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:54.463451 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6fk95" event={"ID":"0778a52f-26ea-49d3-b3f6-8ecf5e874e85","Type":"ContainerStarted","Data":"4ba7b761ca668cda32cf7484a29d425880bf79fca59a8de897034b85029b6bff"} Sep 17 07:09:54.505187 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:54.505128 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6fk95" podStartSLOduration=2.516597069 podStartE2EDuration="3.505110266s" totalImagesPullingTime="924.596442ms" totalInitContainerRuntime="63.916755ms" isStatefulPod=true podCreationTimestamp="2026-09-17 07:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:09:54.504007277 +0000 UTC m=+106.996082069" watchObservedRunningTime="2026-09-17 07:09:54.505110266 +0000 UTC m=+106.997185058" Sep 17 07:09:54.620338 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:54.620280 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:55.713022 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:55.712990 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:55.843372 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:55.843339 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:55.930562 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:55.930533 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:56.396496 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:56.396463 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:56.540755 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:56.540727 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:57.302131 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:57.302050 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:58.442461 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:58.442431 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-79g8v" Sep 17 07:09:58.947370 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:58.947338 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:09:58.958224 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:58.958199 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:09:59.261171 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:09:59.261079 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:00.198580 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:00.198507 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:00.298570 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:00.298542 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:00.407108 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:00.407071 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:00.503805 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:00.503713 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:00.765897 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:00.765818 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:00.891456 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:00.891423 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:01.116372 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:01.116343 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:01.225494 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:01.225428 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:03.442923 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:03.442888 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:03.521254 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:03.521224 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:03.600266 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:03.600236 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:03.971888 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:03.971852 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" podUID="188de529-94f1-4a15-900e-fade80cfa037" containerName="registry" containerID="cri-o://d8aea0cc13e63858bf845f75030c62e42840daa7e10870b239d78e6896cb17fc" gracePeriod=30 Sep 17 07:10:04.208921 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.208896 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:10:04.300196 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.300170 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/188de529-94f1-4a15-900e-fade80cfa037-trusted-ca\") pod \"188de529-94f1-4a15-900e-fade80cfa037\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " Sep 17 07:10:04.300360 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.300212 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/188de529-94f1-4a15-900e-fade80cfa037-registry-certificates\") pod \"188de529-94f1-4a15-900e-fade80cfa037\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " Sep 17 07:10:04.300360 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.300236 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-bound-sa-token\") pod \"188de529-94f1-4a15-900e-fade80cfa037\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " Sep 17 07:10:04.300360 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.300260 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/188de529-94f1-4a15-900e-fade80cfa037-ca-trust-extracted\") pod \"188de529-94f1-4a15-900e-fade80cfa037\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " Sep 17 07:10:04.300360 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.300276 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls\") pod \"188de529-94f1-4a15-900e-fade80cfa037\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " Sep 17 07:10:04.300360 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.300319 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/188de529-94f1-4a15-900e-fade80cfa037-image-registry-private-configuration\") pod \"188de529-94f1-4a15-900e-fade80cfa037\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " Sep 17 07:10:04.300360 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.300353 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/188de529-94f1-4a15-900e-fade80cfa037-installation-pull-secrets\") pod \"188de529-94f1-4a15-900e-fade80cfa037\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " Sep 17 07:10:04.300640 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.300396 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94dkb\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-kube-api-access-94dkb\") pod \"188de529-94f1-4a15-900e-fade80cfa037\" (UID: \"188de529-94f1-4a15-900e-fade80cfa037\") " Sep 17 07:10:04.300906 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.300757 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/188de529-94f1-4a15-900e-fade80cfa037-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "188de529-94f1-4a15-900e-fade80cfa037" (UID: "188de529-94f1-4a15-900e-fade80cfa037"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:10:04.300906 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.300820 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/188de529-94f1-4a15-900e-fade80cfa037-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "188de529-94f1-4a15-900e-fade80cfa037" (UID: "188de529-94f1-4a15-900e-fade80cfa037"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:10:04.303209 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.303181 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "188de529-94f1-4a15-900e-fade80cfa037" (UID: "188de529-94f1-4a15-900e-fade80cfa037"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:10:04.303209 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.303198 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/188de529-94f1-4a15-900e-fade80cfa037-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "188de529-94f1-4a15-900e-fade80cfa037" (UID: "188de529-94f1-4a15-900e-fade80cfa037"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:10:04.303379 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.303224 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "188de529-94f1-4a15-900e-fade80cfa037" (UID: "188de529-94f1-4a15-900e-fade80cfa037"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:10:04.303379 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.303265 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/188de529-94f1-4a15-900e-fade80cfa037-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "188de529-94f1-4a15-900e-fade80cfa037" (UID: "188de529-94f1-4a15-900e-fade80cfa037"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:10:04.303379 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.303319 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-kube-api-access-94dkb" (OuterVolumeSpecName: "kube-api-access-94dkb") pod "188de529-94f1-4a15-900e-fade80cfa037" (UID: "188de529-94f1-4a15-900e-fade80cfa037"). InnerVolumeSpecName "kube-api-access-94dkb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:10:04.308553 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.308528 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/188de529-94f1-4a15-900e-fade80cfa037-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "188de529-94f1-4a15-900e-fade80cfa037" (UID: "188de529-94f1-4a15-900e-fade80cfa037"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:10:04.401060 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.401023 2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/188de529-94f1-4a15-900e-fade80cfa037-image-registry-private-configuration\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:10:04.401060 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.401053 2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/188de529-94f1-4a15-900e-fade80cfa037-installation-pull-secrets\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:10:04.401060 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.401064 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94dkb\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-kube-api-access-94dkb\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:10:04.401284 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.401073 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/188de529-94f1-4a15-900e-fade80cfa037-trusted-ca\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:10:04.401284 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.401082 2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/188de529-94f1-4a15-900e-fade80cfa037-registry-certificates\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:10:04.401284 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.401092 2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-bound-sa-token\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:10:04.401284 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.401100 2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/188de529-94f1-4a15-900e-fade80cfa037-ca-trust-extracted\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:10:04.401284 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.401108 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/188de529-94f1-4a15-900e-fade80cfa037-registry-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:10:04.500770 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.500732 2566 generic.go:356] "Generic (PLEG): container finished" podID="188de529-94f1-4a15-900e-fade80cfa037" containerID="d8aea0cc13e63858bf845f75030c62e42840daa7e10870b239d78e6896cb17fc" exitCode=0 Sep 17 07:10:04.501196 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.500794 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" Sep 17 07:10:04.501196 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.500823 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" event={"ID":"188de529-94f1-4a15-900e-fade80cfa037","Type":"ContainerDied","Data":"d8aea0cc13e63858bf845f75030c62e42840daa7e10870b239d78e6896cb17fc"} Sep 17 07:10:04.501196 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.500873 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-789fdfb4f-bhdgd" event={"ID":"188de529-94f1-4a15-900e-fade80cfa037","Type":"ContainerDied","Data":"e3d2668ec9913796abe2fb29e486a2f6e4418b256520969bd7bd8d0d2bb1713f"} Sep 17 07:10:04.501196 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.500894 2566 scope.go:117] "RemoveContainer" containerID="d8aea0cc13e63858bf845f75030c62e42840daa7e10870b239d78e6896cb17fc" Sep 17 07:10:04.509460 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.509444 2566 scope.go:117] "RemoveContainer" containerID="d8aea0cc13e63858bf845f75030c62e42840daa7e10870b239d78e6896cb17fc" Sep 17 07:10:04.509720 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:10:04.509701 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8aea0cc13e63858bf845f75030c62e42840daa7e10870b239d78e6896cb17fc\": container with ID starting with d8aea0cc13e63858bf845f75030c62e42840daa7e10870b239d78e6896cb17fc not found: ID does not exist" containerID="d8aea0cc13e63858bf845f75030c62e42840daa7e10870b239d78e6896cb17fc" Sep 17 07:10:04.509794 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.509727 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8aea0cc13e63858bf845f75030c62e42840daa7e10870b239d78e6896cb17fc"} err="failed to get container status \"d8aea0cc13e63858bf845f75030c62e42840daa7e10870b239d78e6896cb17fc\": rpc error: code = NotFound desc = could not find container \"d8aea0cc13e63858bf845f75030c62e42840daa7e10870b239d78e6896cb17fc\": container with ID starting with d8aea0cc13e63858bf845f75030c62e42840daa7e10870b239d78e6896cb17fc not found: ID does not exist" Sep 17 07:10:04.522239 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.522215 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-789fdfb4f-bhdgd"] Sep 17 07:10:04.525959 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:04.525940 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-789fdfb4f-bhdgd"] Sep 17 07:10:06.022558 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:06.022526 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="188de529-94f1-4a15-900e-fade80cfa037" path="/var/lib/kubelet/pods/188de529-94f1-4a15-900e-fade80cfa037/volumes" Sep 17 07:10:07.246252 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:07.246222 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:07.357024 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:07.356989 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:07.481765 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:07.481731 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:08.866555 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:08.866519 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:08.978803 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:08.978746 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:09.145341 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:09.145252 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:09.247869 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:09.247837 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:10.018201 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:10.018167 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:10.125346 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:10.125313 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:12.222946 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:12.222889 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:12.344694 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:12.344666 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:15.694410 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:15.694366 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:15.823340 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:15.823304 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:16.897886 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:16.897850 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs\") pod \"network-metrics-daemon-z2l2s\" (UID: \"dc4c353d-0b67-4b66-a052-dd8397466d6f\") " pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:10:16.900203 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:16.900180 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc4c353d-0b67-4b66-a052-dd8397466d6f-metrics-certs\") pod \"network-metrics-daemon-z2l2s\" (UID: \"dc4c353d-0b67-4b66-a052-dd8397466d6f\") " pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:10:17.131920 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:17.131881 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g2nsv\"" Sep 17 07:10:17.139948 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:17.139919 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z2l2s" Sep 17 07:10:17.487908 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:17.487818 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-z2l2s"] Sep 17 07:10:17.491870 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:10:17.491845 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc4c353d_0b67_4b66_a052_dd8397466d6f.slice/crio-0eff9dc038a2684af085501d6f29777c77dedb9553d6ecd2c39c60c00a9536ca WatchSource:0}: Error finding container 0eff9dc038a2684af085501d6f29777c77dedb9553d6ecd2c39c60c00a9536ca: Status 404 returned error can't find the container with id 0eff9dc038a2684af085501d6f29777c77dedb9553d6ecd2c39c60c00a9536ca Sep 17 07:10:17.543859 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:17.543820 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z2l2s" event={"ID":"dc4c353d-0b67-4b66-a052-dd8397466d6f","Type":"ContainerStarted","Data":"0eff9dc038a2684af085501d6f29777c77dedb9553d6ecd2c39c60c00a9536ca"} Sep 17 07:10:18.200328 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:18.200299 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:18.388483 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:18.388448 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:19.111325 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:19.111296 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:19.251004 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:19.250948 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:19.552623 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:19.552588 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z2l2s" event={"ID":"dc4c353d-0b67-4b66-a052-dd8397466d6f","Type":"ContainerStarted","Data":"7e13a151a87a734e9608bb443fdad6d5b22d800348b9f3405ec477166db190c7"} Sep 17 07:10:19.552814 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:19.552651 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z2l2s" event={"ID":"dc4c353d-0b67-4b66-a052-dd8397466d6f","Type":"ContainerStarted","Data":"da186789fae77af6c2a0eeaf107e8b0be4d9953d7fb2660dd5a626e082942579"} Sep 17 07:10:19.582322 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:19.582272 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-z2l2s" podStartSLOduration=130.079468662 podStartE2EDuration="2m11.58225662s" totalImagesPullingTime="1.502787958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:10:19.578317302 +0000 UTC m=+132.070392096" watchObservedRunningTime="2026-09-17 07:10:19.58225662 +0000 UTC m=+132.074331413" Sep 17 07:10:19.906002 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:19.905926 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:20.150355 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:20.150324 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:21.400350 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:21.400313 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:21.639515 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:21.639484 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:21.807321 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:21.807293 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:24.781424 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:24.781385 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:24.943495 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:24.943455 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:30.778856 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:30.778822 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:30.884904 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:30.884861 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:39.812603 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:39.812565 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:39.879578 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:39.879544 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:44.044423 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:44.044388 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:44.145087 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:44.144999 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:10:44.639165 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:44.639135 2566 generic.go:356] "Generic (PLEG): container finished" podID="598a71c4-d028-4460-af48-b76302639301" containerID="6e8fe350ab1c3fb14726157d236ea47eeb421744b4dc8155b9a2982f604e5d92" exitCode=0 Sep 17 07:10:44.639338 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:44.639171 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z" event={"ID":"598a71c4-d028-4460-af48-b76302639301","Type":"ContainerDied","Data":"6e8fe350ab1c3fb14726157d236ea47eeb421744b4dc8155b9a2982f604e5d92"} Sep 17 07:10:44.639462 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:44.639448 2566 scope.go:117] "RemoveContainer" containerID="6e8fe350ab1c3fb14726157d236ea47eeb421744b4dc8155b9a2982f604e5d92" Sep 17 07:10:45.644492 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:10:45.644456 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-txr6z" event={"ID":"598a71c4-d028-4460-af48-b76302639301","Type":"ContainerStarted","Data":"589c68ba37a6d875a8cdae7f03e564ed9490d44b7d78793ea9810efc2c56afb9"} Sep 17 07:11:10.346339 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:11:10.346292 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:11:10.640316 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:11:10.640211 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:11:18.973126 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:11:18.973075 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:11:19.071289 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:11:19.071252 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:11:29.467456 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:11:29.467409 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:11:29.568153 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:11:29.568102 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:11:29.643519 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:11:29.643439 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:12:02.730113 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:12:02.730077 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:12:14.164107 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:12:14.164043 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:12:14.239590 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:12:14.239561 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:12:25.926894 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:12:25.926832 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:12:26.039847 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:12:26.039805 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:12:32.569967 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:12:32.569863 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:12:32.665672 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:12:32.665623 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:12:37.928758 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:12:37.928727 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:12:37.998912 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:12:37.998885 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:12:43.808189 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:12:43.808153 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:12:43.908529 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:12:43.908492 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:12:44.009245 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:12:44.009205 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:12:45.698079 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:12:45.698039 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:12:45.798401 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:12:45.798368 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:13:07.925801 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:13:07.925767 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/1.log" Sep 17 07:13:07.934036 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:13:07.933998 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/1.log" Sep 17 07:13:07.934699 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:13:07.934678 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:13:07.939591 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:13:07.939569 2566 kubelet.go:1648] "Image garbage collection succeeded" Sep 17 07:13:07.944190 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:13:07.944170 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:13:28.811498 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:13:28.811466 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:13:54.778006 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:13:54.777974 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:13:56.014456 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:13:56.014426 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:14:42.310660 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:42.310614 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:14:42.398015 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:42.397959 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:14:45.066997 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:45.066967 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:14:45.160872 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:45.160836 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:14:45.917714 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:45.917646 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:14:45.935713 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:45.935686 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68"] Sep 17 07:14:45.935951 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:45.935940 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="188de529-94f1-4a15-900e-fade80cfa037" containerName="registry" Sep 17 07:14:45.935994 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:45.935954 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="188de529-94f1-4a15-900e-fade80cfa037" containerName="registry" Sep 17 07:14:45.936027 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:45.936004 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="188de529-94f1-4a15-900e-fade80cfa037" containerName="registry" Sep 17 07:14:45.938718 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:45.938703 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68" Sep 17 07:14:45.941017 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:45.940996 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 17 07:14:45.941795 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:45.941764 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-2nmvn\"" Sep 17 07:14:45.941883 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:45.941778 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 17 07:14:45.947879 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:45.947851 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68"] Sep 17 07:14:46.041363 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:46.041332 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4494j\" (UniqueName: \"kubernetes.io/projected/c788b011-77fc-4729-8eda-c04c4369cdae-kube-api-access-4494j\") pod \"feast-operator-controller-manager-77c6578fbc-rkp68\" (UID: \"c788b011-77fc-4729-8eda-c04c4369cdae\") " pod="feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68" Sep 17 07:14:46.142699 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:46.142663 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4494j\" (UniqueName: \"kubernetes.io/projected/c788b011-77fc-4729-8eda-c04c4369cdae-kube-api-access-4494j\") pod \"feast-operator-controller-manager-77c6578fbc-rkp68\" (UID: \"c788b011-77fc-4729-8eda-c04c4369cdae\") " pod="feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68" Sep 17 07:14:46.151028 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:46.150999 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4494j\" (UniqueName: \"kubernetes.io/projected/c788b011-77fc-4729-8eda-c04c4369cdae-kube-api-access-4494j\") pod \"feast-operator-controller-manager-77c6578fbc-rkp68\" (UID: \"c788b011-77fc-4729-8eda-c04c4369cdae\") " pod="feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68" Sep 17 07:14:46.248815 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:46.248700 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68" Sep 17 07:14:46.519491 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:46.519417 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68"] Sep 17 07:14:46.522365 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:14:46.522332 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc788b011_77fc_4729_8eda_c04c4369cdae.slice/crio-e14d47ed5ee82ed9de9660208b541d83da37feb7b2714eb90a22be68dfc296ec WatchSource:0}: Error finding container e14d47ed5ee82ed9de9660208b541d83da37feb7b2714eb90a22be68dfc296ec: Status 404 returned error can't find the container with id e14d47ed5ee82ed9de9660208b541d83da37feb7b2714eb90a22be68dfc296ec Sep 17 07:14:46.524248 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:46.524232 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:14:47.442413 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:47.442367 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68" event={"ID":"c788b011-77fc-4729-8eda-c04c4369cdae","Type":"ContainerStarted","Data":"e14d47ed5ee82ed9de9660208b541d83da37feb7b2714eb90a22be68dfc296ec"} Sep 17 07:14:49.451186 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:49.451156 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68" event={"ID":"c788b011-77fc-4729-8eda-c04c4369cdae","Type":"ContainerStarted","Data":"7c76e143f5af2d2ea6ecc30f486c35181cef07fc6416603b7fefce8f4a2a034c"} Sep 17 07:14:49.451598 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:49.451287 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68" Sep 17 07:14:49.466431 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:49.466389 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68" podStartSLOduration=2.18804721 podStartE2EDuration="4.466371903s" totalImagesPullingTime="2.278324693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:14:49.466084833 +0000 UTC m=+401.958159627" watchObservedRunningTime="2026-09-17 07:14:49.466371903 +0000 UTC m=+401.958446697" Sep 17 07:14:53.212163 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:53.212128 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:14:53.313799 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:53.313755 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:14:53.377195 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:14:53.377161 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:15:00.457387 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:15:00.457357 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68" Sep 17 07:15:12.830062 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:15:12.830023 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:15:23.606636 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:15:23.606600 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:15:23.706829 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:15:23.706778 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:15:24.199117 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:15:24.199086 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:15:24.297374 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:15:24.297344 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:15:24.386973 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:15:24.386942 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:15:24.453574 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:15:24.453492 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:15:24.553378 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:15:24.553346 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:15:31.631312 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:15:31.631233 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:15:53.322223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:15:53.322189 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:15:53.412464 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:15:53.412432 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:16:13.398449 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:16:13.398415 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:16:31.271175 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:16:31.271139 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:17:14.887980 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:17:14.887948 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:17:15.222393 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:17:15.222312 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:17:15.417475 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:17:15.417444 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:17:15.580973 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:17:15.580939 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:17:15.846157 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:17:15.846077 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:17:37.959847 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:17:37.959818 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:18:07.951243 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:18:07.951216 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/1.log" Sep 17 07:18:07.956057 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:18:07.956033 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/1.log" Sep 17 07:18:07.958112 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:18:07.958093 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:18:07.962947 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:18:07.962930 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:18:39.953930 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:18:39.953838 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:18:40.054124 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:18:40.054091 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:18:40.396412 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:18:40.396376 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:18:40.499120 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:18:40.499084 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:18:40.628895 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:18:40.628862 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:18:40.794507 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:18:40.794481 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:18:40.916086 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:18:40.916055 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:19:24.293431 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:19:24.293393 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:20:23.673598 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:20:23.673566 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:20:26.861122 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:20:26.861087 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:20:26.965462 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:20:26.965426 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:20:27.037628 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:20:27.037595 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:20:27.105205 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:20:27.105175 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:20:32.150756 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:20:32.150717 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:21:09.908079 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:09.908045 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:21:10.132682 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.132631 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-68d7q"] Sep 17 07:21:10.135762 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.135742 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-68d7q" Sep 17 07:21:10.138639 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.138618 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 17 07:21:10.138893 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.138871 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 07:21:10.138983 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.138970 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 07:21:10.139663 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.139635 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-qcmsg\"" Sep 17 07:21:10.144558 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.144538 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-68d7q"] Sep 17 07:21:10.177851 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.177751 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:21:10.228948 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.228914 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr4h9\" (UniqueName: \"kubernetes.io/projected/f28f7ef9-7b1a-48dc-8460-ee344cd104c7-kube-api-access-dr4h9\") pod \"postgres-864846d4f5-68d7q\" (UID: \"f28f7ef9-7b1a-48dc-8460-ee344cd104c7\") " pod="test-ns-feast/postgres-864846d4f5-68d7q" Sep 17 07:21:10.228948 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.228950 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/f28f7ef9-7b1a-48dc-8460-ee344cd104c7-postgresdata\") pod \"postgres-864846d4f5-68d7q\" (UID: \"f28f7ef9-7b1a-48dc-8460-ee344cd104c7\") " pod="test-ns-feast/postgres-864846d4f5-68d7q" Sep 17 07:21:10.253979 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.253955 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:21:10.330237 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.330206 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr4h9\" (UniqueName: \"kubernetes.io/projected/f28f7ef9-7b1a-48dc-8460-ee344cd104c7-kube-api-access-dr4h9\") pod \"postgres-864846d4f5-68d7q\" (UID: \"f28f7ef9-7b1a-48dc-8460-ee344cd104c7\") " pod="test-ns-feast/postgres-864846d4f5-68d7q" Sep 17 07:21:10.330321 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.330245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/f28f7ef9-7b1a-48dc-8460-ee344cd104c7-postgresdata\") pod \"postgres-864846d4f5-68d7q\" (UID: \"f28f7ef9-7b1a-48dc-8460-ee344cd104c7\") " pod="test-ns-feast/postgres-864846d4f5-68d7q" Sep 17 07:21:10.330657 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.330639 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/f28f7ef9-7b1a-48dc-8460-ee344cd104c7-postgresdata\") pod \"postgres-864846d4f5-68d7q\" (UID: \"f28f7ef9-7b1a-48dc-8460-ee344cd104c7\") " pod="test-ns-feast/postgres-864846d4f5-68d7q" Sep 17 07:21:10.338627 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.338597 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr4h9\" (UniqueName: \"kubernetes.io/projected/f28f7ef9-7b1a-48dc-8460-ee344cd104c7-kube-api-access-dr4h9\") pod \"postgres-864846d4f5-68d7q\" (UID: \"f28f7ef9-7b1a-48dc-8460-ee344cd104c7\") " pod="test-ns-feast/postgres-864846d4f5-68d7q" Sep 17 07:21:10.444390 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.444310 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-68d7q" Sep 17 07:21:10.711664 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.711590 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-68d7q"] Sep 17 07:21:10.714814 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:21:10.714774 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf28f7ef9_7b1a_48dc_8460_ee344cd104c7.slice/crio-731cc7c8fabd8459d555d42884e7ccd3e06f2bf0e621ff7f00651496247e397c WatchSource:0}: Error finding container 731cc7c8fabd8459d555d42884e7ccd3e06f2bf0e621ff7f00651496247e397c: Status 404 returned error can't find the container with id 731cc7c8fabd8459d555d42884e7ccd3e06f2bf0e621ff7f00651496247e397c Sep 17 07:21:10.716955 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:10.716934 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:21:11.709170 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:11.709135 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-68d7q" event={"ID":"f28f7ef9-7b1a-48dc-8460-ee344cd104c7","Type":"ContainerStarted","Data":"731cc7c8fabd8459d555d42884e7ccd3e06f2bf0e621ff7f00651496247e397c"} Sep 17 07:21:18.737008 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:18.736927 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-68d7q" event={"ID":"f28f7ef9-7b1a-48dc-8460-ee344cd104c7","Type":"ContainerStarted","Data":"942aaf96b886ea63498b223eea2fc579f4b9c93a369549137e47ac8a245f9e26"} Sep 17 07:21:18.752712 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:18.752638 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-68d7q" podStartSLOduration=1.152499193 podStartE2EDuration="8.752617741s" totalImagesPullingTime="7.600118548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:21:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:21:18.751038784 +0000 UTC m=+791.243113577" watchObservedRunningTime="2026-09-17 07:21:18.752617741 +0000 UTC m=+791.244692535" Sep 17 07:21:23.012940 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:23.012880 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:21:23.169965 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:23.169924 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:21:23.330119 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:23.330072 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:21:23.434802 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:23.434751 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:21:37.690028 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:37.689947 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:21:48.042661 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:21:48.042628 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:22:48.846993 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:48.846964 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:22:53.835074 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:53.835041 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-68d7q"] Sep 17 07:22:53.835522 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:53.835203 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-68d7q" podUID="f28f7ef9-7b1a-48dc-8460-ee344cd104c7" containerName="postgres" containerID="cri-o://942aaf96b886ea63498b223eea2fc579f4b9c93a369549137e47ac8a245f9e26" gracePeriod=30 Sep 17 07:22:53.915521 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:53.915492 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:22:54.057170 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:54.057143 2566 generic.go:356] "Generic (PLEG): container finished" podID="f28f7ef9-7b1a-48dc-8460-ee344cd104c7" containerID="942aaf96b886ea63498b223eea2fc579f4b9c93a369549137e47ac8a245f9e26" exitCode=0 Sep 17 07:22:54.057301 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:54.057202 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-68d7q" event={"ID":"f28f7ef9-7b1a-48dc-8460-ee344cd104c7","Type":"ContainerDied","Data":"942aaf96b886ea63498b223eea2fc579f4b9c93a369549137e47ac8a245f9e26"} Sep 17 07:22:54.070634 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:54.070607 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-68d7q" Sep 17 07:22:54.167935 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:54.167854 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dr4h9\" (UniqueName: \"kubernetes.io/projected/f28f7ef9-7b1a-48dc-8460-ee344cd104c7-kube-api-access-dr4h9\") pod \"f28f7ef9-7b1a-48dc-8460-ee344cd104c7\" (UID: \"f28f7ef9-7b1a-48dc-8460-ee344cd104c7\") " Sep 17 07:22:54.167935 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:54.167928 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/f28f7ef9-7b1a-48dc-8460-ee344cd104c7-postgresdata\") pod \"f28f7ef9-7b1a-48dc-8460-ee344cd104c7\" (UID: \"f28f7ef9-7b1a-48dc-8460-ee344cd104c7\") " Sep 17 07:22:54.170186 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:54.170150 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f28f7ef9-7b1a-48dc-8460-ee344cd104c7-kube-api-access-dr4h9" (OuterVolumeSpecName: "kube-api-access-dr4h9") pod "f28f7ef9-7b1a-48dc-8460-ee344cd104c7" (UID: "f28f7ef9-7b1a-48dc-8460-ee344cd104c7"). InnerVolumeSpecName "kube-api-access-dr4h9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:22:54.198797 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:54.198750 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f28f7ef9-7b1a-48dc-8460-ee344cd104c7-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "f28f7ef9-7b1a-48dc-8460-ee344cd104c7" (UID: "f28f7ef9-7b1a-48dc-8460-ee344cd104c7"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:22:54.269061 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:54.269015 2566 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/f28f7ef9-7b1a-48dc-8460-ee344cd104c7-postgresdata\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:22:54.269061 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:54.269055 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dr4h9\" (UniqueName: \"kubernetes.io/projected/f28f7ef9-7b1a-48dc-8460-ee344cd104c7-kube-api-access-dr4h9\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:22:55.061633 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:55.061598 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-68d7q" event={"ID":"f28f7ef9-7b1a-48dc-8460-ee344cd104c7","Type":"ContainerDied","Data":"731cc7c8fabd8459d555d42884e7ccd3e06f2bf0e621ff7f00651496247e397c"} Sep 17 07:22:55.061633 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:55.061633 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-68d7q" Sep 17 07:22:55.062115 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:55.061649 2566 scope.go:117] "RemoveContainer" containerID="942aaf96b886ea63498b223eea2fc579f4b9c93a369549137e47ac8a245f9e26" Sep 17 07:22:55.112665 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:55.112626 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-68d7q"] Sep 17 07:22:55.118979 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:55.118959 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-68d7q"] Sep 17 07:22:56.021804 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:22:56.021758 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f28f7ef9-7b1a-48dc-8460-ee344cd104c7" path="/var/lib/kubelet/pods/f28f7ef9-7b1a-48dc-8460-ee344cd104c7/volumes" Sep 17 07:23:07.971806 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:23:07.971756 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/1.log" Sep 17 07:23:07.974323 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:23:07.974294 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/1.log" Sep 17 07:23:07.978728 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:23:07.978707 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:23:07.985970 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:23:07.985948 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:23:31.487131 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:23:31.487087 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:23:31.562019 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:23:31.561976 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:23:32.457981 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:23:32.457945 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:23:32.623470 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:23:32.623435 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:24:58.041183 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:24:58.041101 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:24:59.511446 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:24:59.511412 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:25:33.712378 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:33.712338 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:25:42.961414 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:42.961377 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:25:43.062142 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:43.062108 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:25:43.173390 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:43.173355 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:25:48.610302 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.610273 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-nk4kc"] Sep 17 07:25:48.610659 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.610510 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f28f7ef9-7b1a-48dc-8460-ee344cd104c7" containerName="postgres" Sep 17 07:25:48.610659 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.610520 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f28f7ef9-7b1a-48dc-8460-ee344cd104c7" containerName="postgres" Sep 17 07:25:48.610659 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.610561 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f28f7ef9-7b1a-48dc-8460-ee344cd104c7" containerName="postgres" Sep 17 07:25:48.613234 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.613214 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-nk4kc" Sep 17 07:25:48.616016 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.615992 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 17 07:25:48.616137 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.615994 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 17 07:25:48.616665 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.616647 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-ncmg2\"" Sep 17 07:25:48.616755 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.616716 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 17 07:25:48.630268 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.630246 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-nk4kc"] Sep 17 07:25:48.670450 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.670421 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:25:48.801299 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.801243 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/9b44198e-f727-423c-8ed4-72cb1f2a50bd-postgresdata\") pod \"postgres-864846d4f5-nk4kc\" (UID: \"9b44198e-f727-423c-8ed4-72cb1f2a50bd\") " pod="test-ns-feast-rest/postgres-864846d4f5-nk4kc" Sep 17 07:25:48.801488 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.801342 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n8kp\" (UniqueName: \"kubernetes.io/projected/9b44198e-f727-423c-8ed4-72cb1f2a50bd-kube-api-access-4n8kp\") pod \"postgres-864846d4f5-nk4kc\" (UID: \"9b44198e-f727-423c-8ed4-72cb1f2a50bd\") " pod="test-ns-feast-rest/postgres-864846d4f5-nk4kc" Sep 17 07:25:48.902071 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.901969 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/9b44198e-f727-423c-8ed4-72cb1f2a50bd-postgresdata\") pod \"postgres-864846d4f5-nk4kc\" (UID: \"9b44198e-f727-423c-8ed4-72cb1f2a50bd\") " pod="test-ns-feast-rest/postgres-864846d4f5-nk4kc" Sep 17 07:25:48.902071 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.902040 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n8kp\" (UniqueName: \"kubernetes.io/projected/9b44198e-f727-423c-8ed4-72cb1f2a50bd-kube-api-access-4n8kp\") pod \"postgres-864846d4f5-nk4kc\" (UID: \"9b44198e-f727-423c-8ed4-72cb1f2a50bd\") " pod="test-ns-feast-rest/postgres-864846d4f5-nk4kc" Sep 17 07:25:48.902370 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.902349 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/9b44198e-f727-423c-8ed4-72cb1f2a50bd-postgresdata\") pod \"postgres-864846d4f5-nk4kc\" (UID: \"9b44198e-f727-423c-8ed4-72cb1f2a50bd\") " pod="test-ns-feast-rest/postgres-864846d4f5-nk4kc" Sep 17 07:25:48.910726 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.910705 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n8kp\" (UniqueName: \"kubernetes.io/projected/9b44198e-f727-423c-8ed4-72cb1f2a50bd-kube-api-access-4n8kp\") pod \"postgres-864846d4f5-nk4kc\" (UID: \"9b44198e-f727-423c-8ed4-72cb1f2a50bd\") " pod="test-ns-feast-rest/postgres-864846d4f5-nk4kc" Sep 17 07:25:48.922693 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:48.922669 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-nk4kc" Sep 17 07:25:49.189553 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:49.189471 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-nk4kc"] Sep 17 07:25:49.193419 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:25:49.193395 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b44198e_f727_423c_8ed4_72cb1f2a50bd.slice/crio-6af350e2cc5b6d1eea1c94b8d814c7d878bc66ddda68ebae92a21e3ec5bfc1fc WatchSource:0}: Error finding container 6af350e2cc5b6d1eea1c94b8d814c7d878bc66ddda68ebae92a21e3ec5bfc1fc: Status 404 returned error can't find the container with id 6af350e2cc5b6d1eea1c94b8d814c7d878bc66ddda68ebae92a21e3ec5bfc1fc Sep 17 07:25:49.632758 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:49.632714 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-nk4kc" event={"ID":"9b44198e-f727-423c-8ed4-72cb1f2a50bd","Type":"ContainerStarted","Data":"7c4d8a630c43ac82b30a7affae9ffdf2a165f4a6b3afa4a6add0ae6f805beb56"} Sep 17 07:25:49.632758 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:49.632759 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-nk4kc" event={"ID":"9b44198e-f727-423c-8ed4-72cb1f2a50bd","Type":"ContainerStarted","Data":"6af350e2cc5b6d1eea1c94b8d814c7d878bc66ddda68ebae92a21e3ec5bfc1fc"} Sep 17 07:25:54.192529 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:54.192503 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:25:54.294474 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:54.294441 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:25:54.398925 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:54.398892 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:25:54.468553 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:25:54.468474 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:26:18.960280 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:26:18.960247 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:26:20.086098 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:26:20.086068 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:26:20.190103 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:26:20.190073 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:26:20.316997 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:26:20.316962 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:26:20.511478 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:26:20.511394 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:27:57.717622 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:27:57.717569 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:06.587280 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:06.587241 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:07.998011 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:07.997982 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/1.log" Sep 17 07:28:07.998591 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:07.998577 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/1.log" Sep 17 07:28:08.004941 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:08.004919 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:28:08.005114 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:08.005097 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:28:09.432223 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:09.432183 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:09.899045 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:09.898989 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-nk4kc" podStartSLOduration=141.898968267 podStartE2EDuration="2m21.898968267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:25:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:25:49.664862952 +0000 UTC m=+1062.156937743" watchObservedRunningTime="2026-09-17 07:28:09.898968267 +0000 UTC m=+1202.391043065" Sep 17 07:28:09.899226 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:09.899120 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68"] Sep 17 07:28:09.899343 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:09.899324 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68" podUID="c788b011-77fc-4729-8eda-c04c4369cdae" containerName="manager" containerID="cri-o://7c76e143f5af2d2ea6ecc30f486c35181cef07fc6416603b7fefce8f4a2a034c" gracePeriod=10 Sep 17 07:28:09.924438 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:09.924405 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:10.093846 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:10.093806 2566 generic.go:356] "Generic (PLEG): container finished" podID="c788b011-77fc-4729-8eda-c04c4369cdae" containerID="7c76e143f5af2d2ea6ecc30f486c35181cef07fc6416603b7fefce8f4a2a034c" exitCode=0 Sep 17 07:28:10.093846 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:10.093816 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68" event={"ID":"c788b011-77fc-4729-8eda-c04c4369cdae","Type":"ContainerDied","Data":"7c76e143f5af2d2ea6ecc30f486c35181cef07fc6416603b7fefce8f4a2a034c"} Sep 17 07:28:10.133489 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:10.133465 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68" Sep 17 07:28:10.280527 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:10.280444 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:10.287639 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:10.287615 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4494j\" (UniqueName: \"kubernetes.io/projected/c788b011-77fc-4729-8eda-c04c4369cdae-kube-api-access-4494j\") pod \"c788b011-77fc-4729-8eda-c04c4369cdae\" (UID: \"c788b011-77fc-4729-8eda-c04c4369cdae\") " Sep 17 07:28:10.292939 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:10.292897 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c788b011-77fc-4729-8eda-c04c4369cdae-kube-api-access-4494j" (OuterVolumeSpecName: "kube-api-access-4494j") pod "c788b011-77fc-4729-8eda-c04c4369cdae" (UID: "c788b011-77fc-4729-8eda-c04c4369cdae"). InnerVolumeSpecName "kube-api-access-4494j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:28:10.388860 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:10.388795 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4494j\" (UniqueName: \"kubernetes.io/projected/c788b011-77fc-4729-8eda-c04c4369cdae-kube-api-access-4494j\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:28:10.400299 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:10.400257 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:11.098383 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:11.098355 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68" Sep 17 07:28:11.098805 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:11.098354 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68" event={"ID":"c788b011-77fc-4729-8eda-c04c4369cdae","Type":"ContainerDied","Data":"e14d47ed5ee82ed9de9660208b541d83da37feb7b2714eb90a22be68dfc296ec"} Sep 17 07:28:11.099428 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:11.098478 2566 scope.go:117] "RemoveContainer" containerID="7c76e143f5af2d2ea6ecc30f486c35181cef07fc6416603b7fefce8f4a2a034c" Sep 17 07:28:11.118029 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:11.118003 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68"] Sep 17 07:28:11.123506 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:11.123485 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-77c6578fbc-rkp68"] Sep 17 07:28:11.571764 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:11.571728 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-nk4kc"] Sep 17 07:28:11.571958 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:11.571935 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-nk4kc" podUID="9b44198e-f727-423c-8ed4-72cb1f2a50bd" containerName="postgres" containerID="cri-o://7c4d8a630c43ac82b30a7affae9ffdf2a165f4a6b3afa4a6add0ae6f805beb56" gracePeriod=30 Sep 17 07:28:11.694360 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:11.694330 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:11.805041 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:11.805019 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-nk4kc" Sep 17 07:28:11.899310 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:11.899232 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4n8kp\" (UniqueName: \"kubernetes.io/projected/9b44198e-f727-423c-8ed4-72cb1f2a50bd-kube-api-access-4n8kp\") pod \"9b44198e-f727-423c-8ed4-72cb1f2a50bd\" (UID: \"9b44198e-f727-423c-8ed4-72cb1f2a50bd\") " Sep 17 07:28:11.899310 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:11.899280 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/9b44198e-f727-423c-8ed4-72cb1f2a50bd-postgresdata\") pod \"9b44198e-f727-423c-8ed4-72cb1f2a50bd\" (UID: \"9b44198e-f727-423c-8ed4-72cb1f2a50bd\") " Sep 17 07:28:11.901390 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:11.901356 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b44198e-f727-423c-8ed4-72cb1f2a50bd-kube-api-access-4n8kp" (OuterVolumeSpecName: "kube-api-access-4n8kp") pod "9b44198e-f727-423c-8ed4-72cb1f2a50bd" (UID: "9b44198e-f727-423c-8ed4-72cb1f2a50bd"). InnerVolumeSpecName "kube-api-access-4n8kp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:28:11.929272 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:11.929236 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b44198e-f727-423c-8ed4-72cb1f2a50bd-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "9b44198e-f727-423c-8ed4-72cb1f2a50bd" (UID: "9b44198e-f727-423c-8ed4-72cb1f2a50bd"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:28:12.000305 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:12.000269 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4n8kp\" (UniqueName: \"kubernetes.io/projected/9b44198e-f727-423c-8ed4-72cb1f2a50bd-kube-api-access-4n8kp\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:28:12.000305 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:12.000299 2566 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/9b44198e-f727-423c-8ed4-72cb1f2a50bd-postgresdata\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:28:12.022549 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:12.022519 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c788b011-77fc-4729-8eda-c04c4369cdae" path="/var/lib/kubelet/pods/c788b011-77fc-4729-8eda-c04c4369cdae/volumes" Sep 17 07:28:12.102568 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:12.102535 2566 generic.go:356] "Generic (PLEG): container finished" podID="9b44198e-f727-423c-8ed4-72cb1f2a50bd" containerID="7c4d8a630c43ac82b30a7affae9ffdf2a165f4a6b3afa4a6add0ae6f805beb56" exitCode=0 Sep 17 07:28:12.103041 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:12.102587 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-nk4kc" Sep 17 07:28:12.103041 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:12.102613 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-nk4kc" event={"ID":"9b44198e-f727-423c-8ed4-72cb1f2a50bd","Type":"ContainerDied","Data":"7c4d8a630c43ac82b30a7affae9ffdf2a165f4a6b3afa4a6add0ae6f805beb56"} Sep 17 07:28:12.103041 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:12.102650 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-nk4kc" event={"ID":"9b44198e-f727-423c-8ed4-72cb1f2a50bd","Type":"ContainerDied","Data":"6af350e2cc5b6d1eea1c94b8d814c7d878bc66ddda68ebae92a21e3ec5bfc1fc"} Sep 17 07:28:12.103041 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:12.102666 2566 scope.go:117] "RemoveContainer" containerID="7c4d8a630c43ac82b30a7affae9ffdf2a165f4a6b3afa4a6add0ae6f805beb56" Sep 17 07:28:12.110191 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:12.110170 2566 scope.go:117] "RemoveContainer" containerID="7c4d8a630c43ac82b30a7affae9ffdf2a165f4a6b3afa4a6add0ae6f805beb56" Sep 17 07:28:12.110476 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:28:12.110460 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c4d8a630c43ac82b30a7affae9ffdf2a165f4a6b3afa4a6add0ae6f805beb56\": container with ID starting with 7c4d8a630c43ac82b30a7affae9ffdf2a165f4a6b3afa4a6add0ae6f805beb56 not found: ID does not exist" containerID="7c4d8a630c43ac82b30a7affae9ffdf2a165f4a6b3afa4a6add0ae6f805beb56" Sep 17 07:28:12.110532 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:12.110481 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c4d8a630c43ac82b30a7affae9ffdf2a165f4a6b3afa4a6add0ae6f805beb56"} err="failed to get container status \"7c4d8a630c43ac82b30a7affae9ffdf2a165f4a6b3afa4a6add0ae6f805beb56\": rpc error: code = NotFound desc = could not find container \"7c4d8a630c43ac82b30a7affae9ffdf2a165f4a6b3afa4a6add0ae6f805beb56\": container with ID starting with 7c4d8a630c43ac82b30a7affae9ffdf2a165f4a6b3afa4a6add0ae6f805beb56 not found: ID does not exist" Sep 17 07:28:12.120586 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:12.120563 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-nk4kc"] Sep 17 07:28:12.124648 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:12.124624 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-nk4kc"] Sep 17 07:28:14.021884 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:14.021851 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b44198e-f727-423c-8ed4-72cb1f2a50bd" path="/var/lib/kubelet/pods/9b44198e-f727-423c-8ed4-72cb1f2a50bd/volumes" Sep 17 07:28:15.218121 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:15.218089 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:15.306037 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:15.306004 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:20.794881 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:20.794838 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:20.891137 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:20.891109 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:21.814140 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:21.814106 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:21.829307 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:21.829279 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t"] Sep 17 07:28:21.829530 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:21.829519 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c788b011-77fc-4729-8eda-c04c4369cdae" containerName="manager" Sep 17 07:28:21.829571 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:21.829532 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c788b011-77fc-4729-8eda-c04c4369cdae" containerName="manager" Sep 17 07:28:21.829571 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:21.829546 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b44198e-f727-423c-8ed4-72cb1f2a50bd" containerName="postgres" Sep 17 07:28:21.829571 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:21.829551 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b44198e-f727-423c-8ed4-72cb1f2a50bd" containerName="postgres" Sep 17 07:28:21.829651 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:21.829595 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c788b011-77fc-4729-8eda-c04c4369cdae" containerName="manager" Sep 17 07:28:21.829651 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:21.829602 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b44198e-f727-423c-8ed4-72cb1f2a50bd" containerName="postgres" Sep 17 07:28:21.832524 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:21.832508 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t" Sep 17 07:28:21.839479 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:21.839455 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-jhv5r\"" Sep 17 07:28:21.839615 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:21.839600 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 17 07:28:21.840288 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:21.840274 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 17 07:28:21.871816 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:21.871771 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t"] Sep 17 07:28:21.969343 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:21.969304 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw8w7\" (UniqueName: \"kubernetes.io/projected/b338bf76-721e-4a76-9287-a1c4f5aaf95d-kube-api-access-vw8w7\") pod \"feast-operator-controller-manager-689b877b4b-5pr6t\" (UID: \"b338bf76-721e-4a76-9287-a1c4f5aaf95d\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t" Sep 17 07:28:22.070287 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:22.070195 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw8w7\" (UniqueName: \"kubernetes.io/projected/b338bf76-721e-4a76-9287-a1c4f5aaf95d-kube-api-access-vw8w7\") pod \"feast-operator-controller-manager-689b877b4b-5pr6t\" (UID: \"b338bf76-721e-4a76-9287-a1c4f5aaf95d\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t" Sep 17 07:28:22.103716 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:22.103692 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw8w7\" (UniqueName: \"kubernetes.io/projected/b338bf76-721e-4a76-9287-a1c4f5aaf95d-kube-api-access-vw8w7\") pod \"feast-operator-controller-manager-689b877b4b-5pr6t\" (UID: \"b338bf76-721e-4a76-9287-a1c4f5aaf95d\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t" Sep 17 07:28:22.141606 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:22.141575 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t" Sep 17 07:28:22.432504 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:22.432429 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t"] Sep 17 07:28:22.435391 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:28:22.435366 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb338bf76_721e_4a76_9287_a1c4f5aaf95d.slice/crio-a028f1b270574043e82750f2fe8eb42a245c4ad9985f65c3641c577b1154139d WatchSource:0}: Error finding container a028f1b270574043e82750f2fe8eb42a245c4ad9985f65c3641c577b1154139d: Status 404 returned error can't find the container with id a028f1b270574043e82750f2fe8eb42a245c4ad9985f65c3641c577b1154139d Sep 17 07:28:22.437282 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:22.437263 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:28:23.139205 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:23.139171 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t" event={"ID":"b338bf76-721e-4a76-9287-a1c4f5aaf95d","Type":"ContainerStarted","Data":"a028f1b270574043e82750f2fe8eb42a245c4ad9985f65c3641c577b1154139d"} Sep 17 07:28:25.147529 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:25.147492 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t" event={"ID":"b338bf76-721e-4a76-9287-a1c4f5aaf95d","Type":"ContainerStarted","Data":"83598caa78a217ec59e6c19d1a2707bd7e0aead64b42283b149f4478833ae21e"} Sep 17 07:28:25.148027 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:25.147585 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t" Sep 17 07:28:36.155171 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:36.155134 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t" Sep 17 07:28:36.177823 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:36.177749 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t" podStartSLOduration=13.216877981 podStartE2EDuration="15.177729698s" totalImagesPullingTime="1.960851717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:25.178084795 +0000 UTC m=+1217.670159586" watchObservedRunningTime="2026-09-17 07:28:36.177729698 +0000 UTC m=+1228.669804490" Sep 17 07:28:38.101406 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.101372 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:38.216150 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.216110 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:38.721462 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.721427 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:38.827185 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.827140 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:38.931668 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.931632 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd"] Sep 17 07:28:38.935052 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.935035 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:38.941382 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.941360 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 07:28:38.942158 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.942129 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 07:28:38.942712 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.942694 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 17 07:28:38.942854 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.942835 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 17 07:28:38.943149 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.943123 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-ftt4z\"" Sep 17 07:28:38.943407 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.943390 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 17 07:28:38.943577 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.943557 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 17 07:28:38.966629 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.966597 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd"] Sep 17 07:28:38.975313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.975228 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:38.989363 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.989337 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/21c5de96-e2d7-49bd-b490-22f0a853126a-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:38.989494 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.989374 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:38.989494 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.989403 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmqvq\" (UniqueName: \"kubernetes.io/projected/21c5de96-e2d7-49bd-b490-22f0a853126a-kube-api-access-pmqvq\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:38.989494 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.989462 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:38.989494 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.989485 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:38.989632 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:38.989517 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:39.090040 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:39.090000 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:39.090040 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:39.090043 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmqvq\" (UniqueName: \"kubernetes.io/projected/21c5de96-e2d7-49bd-b490-22f0a853126a-kube-api-access-pmqvq\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:39.090259 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:39.090082 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:39.090259 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:39.090099 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:39.090259 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:39.090119 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:39.090259 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:39.090137 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/21c5de96-e2d7-49bd-b490-22f0a853126a-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:39.090566 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:39.090533 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/21c5de96-e2d7-49bd-b490-22f0a853126a-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:39.092660 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:39.092625 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:39.092800 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:39.092630 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:39.092800 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:39.092742 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:39.092800 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:39.092751 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:39.105430 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:39.105403 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmqvq\" (UniqueName: \"kubernetes.io/projected/21c5de96-e2d7-49bd-b490-22f0a853126a-kube-api-access-pmqvq\") pod \"feast-simple-feast-setup-57576b8fcf-lrlzd\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:39.149068 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:39.149037 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:39.244388 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:39.244293 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:28:39.288689 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:39.288652 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:39.531944 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:39.531917 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd"] Sep 17 07:28:39.534235 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:28:39.534209 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21c5de96_e2d7_49bd_b490_22f0a853126a.slice/crio-83f2a6d63ff9cfcfe983823ce719b6f71486d3d7e38cdc843f3ffa3394a46e6f WatchSource:0}: Error finding container 83f2a6d63ff9cfcfe983823ce719b6f71486d3d7e38cdc843f3ffa3394a46e6f: Status 404 returned error can't find the container with id 83f2a6d63ff9cfcfe983823ce719b6f71486d3d7e38cdc843f3ffa3394a46e6f Sep 17 07:28:40.201650 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:40.201612 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" event={"ID":"21c5de96-e2d7-49bd-b490-22f0a853126a","Type":"ContainerStarted","Data":"83f2a6d63ff9cfcfe983823ce719b6f71486d3d7e38cdc843f3ffa3394a46e6f"} Sep 17 07:28:49.021808 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:49.021759 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:49.130744 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:49.130711 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:50.140008 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:50.139968 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:54.259262 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:54.259220 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" event={"ID":"21c5de96-e2d7-49bd-b490-22f0a853126a","Type":"ContainerStarted","Data":"0f7eb0b215134d7c94e1ea1bcecd9d6d785717b7b6a04634564976ba67d4a785"} Sep 17 07:28:58.275153 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:58.275121 2566 generic.go:356] "Generic (PLEG): container finished" podID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerID="0f7eb0b215134d7c94e1ea1bcecd9d6d785717b7b6a04634564976ba67d4a785" exitCode=0 Sep 17 07:28:58.275602 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:58.275183 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" event={"ID":"21c5de96-e2d7-49bd-b490-22f0a853126a","Type":"ContainerDied","Data":"0f7eb0b215134d7c94e1ea1bcecd9d6d785717b7b6a04634564976ba67d4a785"} Sep 17 07:28:59.280744 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:28:59.280697 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" event={"ID":"21c5de96-e2d7-49bd-b490-22f0a853126a","Type":"ContainerStarted","Data":"40c3fcd6576da73c95ea9a99ba3b9b6cbd73f69baba85c2a477c441d38fb589b"} Sep 17 07:29:04.301209 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:04.301176 2566 generic.go:356] "Generic (PLEG): container finished" podID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerID="40c3fcd6576da73c95ea9a99ba3b9b6cbd73f69baba85c2a477c441d38fb589b" exitCode=0 Sep 17 07:29:04.301616 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:04.301228 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" event={"ID":"21c5de96-e2d7-49bd-b490-22f0a853126a","Type":"ContainerDied","Data":"40c3fcd6576da73c95ea9a99ba3b9b6cbd73f69baba85c2a477c441d38fb589b"} Sep 17 07:29:05.310449 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:05.310398 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" event={"ID":"21c5de96-e2d7-49bd-b490-22f0a853126a","Type":"ContainerStarted","Data":"0802b51d7f2eaf14a9c22ddbc824de13582a490e44adcd196dbf973e288cb69c"} Sep 17 07:29:05.310943 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:05.310458 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" event={"ID":"21c5de96-e2d7-49bd-b490-22f0a853126a","Type":"ContainerStarted","Data":"4269bc78375557d731dd4509b1b00e96939101935f6392739ff81026db24b046"} Sep 17 07:29:05.310943 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:05.310478 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" event={"ID":"21c5de96-e2d7-49bd-b490-22f0a853126a","Type":"ContainerStarted","Data":"cbe00a71198c1d4e37923058a76cfac409a059fb16f8adc1f4306982ea3613d7"} Sep 17 07:29:05.310943 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:05.310496 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" event={"ID":"21c5de96-e2d7-49bd-b490-22f0a853126a","Type":"ContainerStarted","Data":"387a2a7e2ed7a3ec921570a71cc3bfff72a90b2c4aa1ab7e5dc2b83400f2e377"} Sep 17 07:29:05.339270 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:05.339206 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podStartSLOduration=13.622243732 podStartE2EDuration="27.339181778s" totalImagesPullingTime="13.716938046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:05.336226184 +0000 UTC m=+1257.828300976" watchObservedRunningTime="2026-09-17 07:29:05.339181778 +0000 UTC m=+1257.831256582" Sep 17 07:29:06.245012 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:06.244796 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:29:06.245012 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:06.244865 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:29:06.245012 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:06.244884 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:29:06.245012 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:06.244901 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:29:06.247428 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:06.247262 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.19:8816: connect: connection refused" Sep 17 07:29:06.247428 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:06.247304 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.19:6571: connect: connection refused" Sep 17 07:29:06.247428 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:06.247341 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.19:6567/health\": dial tcp 10.133.0.19:6567: connect: connection refused" Sep 17 07:29:06.247428 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:06.247316 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.19:8443: connect: connection refused" Sep 17 07:29:09.245338 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:09.245293 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:29:09.245833 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:09.245580 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.19:6571: connect: connection refused" Sep 17 07:29:09.245833 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:09.245616 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.19:8816: connect: connection refused" Sep 17 07:29:09.245833 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:09.245699 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.19:6567/health\": dial tcp 10.133.0.19:6567: connect: connection refused" Sep 17 07:29:09.246016 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:09.245939 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:29:09.246016 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:09.245972 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:29:09.246016 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:09.245991 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:29:09.246195 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:09.246167 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.19:8443: connect: connection refused" Sep 17 07:29:12.245466 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:12.245412 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.19:8443: connect: connection refused" Sep 17 07:29:12.246439 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:12.246414 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:29:12.246568 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:12.246557 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:29:12.247204 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:12.247179 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:29:12.247418 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:12.247400 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:29:12.251546 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:12.251527 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:29:12.352692 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:12.352663 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:29:15.245012 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:15.244983 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:29:15.360483 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:15.360449 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:29:29.799372 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:29.799329 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:54.488877 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:29:54.488776 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:30:17.350066 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:17.350029 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:30:17.361326 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:17.361292 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd"] Sep 17 07:30:17.361581 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:17.361551 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="registry" containerID="cri-o://387a2a7e2ed7a3ec921570a71cc3bfff72a90b2c4aa1ab7e5dc2b83400f2e377" gracePeriod=30 Sep 17 07:30:17.361721 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:17.361602 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="offline" containerID="cri-o://4269bc78375557d731dd4509b1b00e96939101935f6392739ff81026db24b046" gracePeriod=30 Sep 17 07:30:17.361721 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:17.361603 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="ui" containerID="cri-o://0802b51d7f2eaf14a9c22ddbc824de13582a490e44adcd196dbf973e288cb69c" gracePeriod=30 Sep 17 07:30:17.361721 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:17.361633 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="online" containerID="cri-o://cbe00a71198c1d4e37923058a76cfac409a059fb16f8adc1f4306982ea3613d7" gracePeriod=30 Sep 17 07:30:17.894269 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:17.894227 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7"] Sep 17 07:30:17.898392 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:17.898371 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:17.903003 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:17.902974 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-fpj5n\"" Sep 17 07:30:17.921842 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:17.921811 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7"] Sep 17 07:30:17.954021 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:17.953984 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrd9p\" (UniqueName: \"kubernetes.io/projected/fb139edb-c8b2-4357-8753-799f76a0dc76-kube-api-access-nrd9p\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:17.954224 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:17.954058 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:17.954224 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:17.954078 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fb139edb-c8b2-4357-8753-799f76a0dc76-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:17.954224 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:17.954097 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:17.954224 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:17.954112 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:17.954224 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:17.954135 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:18.054913 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.054873 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrd9p\" (UniqueName: \"kubernetes.io/projected/fb139edb-c8b2-4357-8753-799f76a0dc76-kube-api-access-nrd9p\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:18.055088 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.054982 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:18.055088 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.055009 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fb139edb-c8b2-4357-8753-799f76a0dc76-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:18.055088 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.055039 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:18.055088 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.055062 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:18.055281 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.055192 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:18.055365 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:30:18.055320 2566 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 17 07:30:18.055423 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:30:18.055389 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-ui-tls podName:fb139edb-c8b2-4357-8753-799f76a0dc76 nodeName:}" failed. No retries permitted until 2026-09-17 07:30:18.555366969 +0000 UTC m=+1331.047441740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-sjtn7" (UID: "fb139edb-c8b2-4357-8753-799f76a0dc76") : secret "feast-simple-feast-setup-ui-tls" not found Sep 17 07:30:18.055556 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.055518 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fb139edb-c8b2-4357-8753-799f76a0dc76-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:18.057610 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.057585 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:18.058010 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.057990 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:18.058077 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.057990 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:18.072313 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.072288 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrd9p\" (UniqueName: \"kubernetes.io/projected/fb139edb-c8b2-4357-8753-799f76a0dc76-kube-api-access-nrd9p\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:18.559628 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.559593 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:18.561925 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.561908 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-sjtn7\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:18.574283 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.574255 2566 generic.go:356] "Generic (PLEG): container finished" podID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerID="0802b51d7f2eaf14a9c22ddbc824de13582a490e44adcd196dbf973e288cb69c" exitCode=0 Sep 17 07:30:18.574283 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.574278 2566 generic.go:356] "Generic (PLEG): container finished" podID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerID="cbe00a71198c1d4e37923058a76cfac409a059fb16f8adc1f4306982ea3613d7" exitCode=0 Sep 17 07:30:18.574455 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.574328 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" event={"ID":"21c5de96-e2d7-49bd-b490-22f0a853126a","Type":"ContainerDied","Data":"0802b51d7f2eaf14a9c22ddbc824de13582a490e44adcd196dbf973e288cb69c"} Sep 17 07:30:18.574455 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.574378 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" event={"ID":"21c5de96-e2d7-49bd-b490-22f0a853126a","Type":"ContainerDied","Data":"cbe00a71198c1d4e37923058a76cfac409a059fb16f8adc1f4306982ea3613d7"} Sep 17 07:30:18.808211 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.808171 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:18.966138 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:18.966106 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:30:19.088715 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:19.088693 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7"] Sep 17 07:30:19.091152 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:30:19.091118 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb139edb_c8b2_4357_8753_799f76a0dc76.slice/crio-1009994a3a8d22e44011b34c0749180205dc4610a96f9931eb2b4f278c9e496d WatchSource:0}: Error finding container 1009994a3a8d22e44011b34c0749180205dc4610a96f9931eb2b4f278c9e496d: Status 404 returned error can't find the container with id 1009994a3a8d22e44011b34c0749180205dc4610a96f9931eb2b4f278c9e496d Sep 17 07:30:19.580142 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:19.580108 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" event={"ID":"fb139edb-c8b2-4357-8753-799f76a0dc76","Type":"ContainerStarted","Data":"cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191"} Sep 17 07:30:19.580142 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:19.580150 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" event={"ID":"fb139edb-c8b2-4357-8753-799f76a0dc76","Type":"ContainerStarted","Data":"1009994a3a8d22e44011b34c0749180205dc4610a96f9931eb2b4f278c9e496d"} Sep 17 07:30:22.350805 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:22.350736 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.19:6567/health\": dial tcp 10.133.0.19:6567: connect: connection refused" Sep 17 07:30:24.599076 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:24.599046 2566 generic.go:356] "Generic (PLEG): container finished" podID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerID="cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191" exitCode=0 Sep 17 07:30:24.599438 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:24.599119 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" event={"ID":"fb139edb-c8b2-4357-8753-799f76a0dc76","Type":"ContainerDied","Data":"cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191"} Sep 17 07:30:25.360861 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:25.360802 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.19:8443: connect: connection refused" Sep 17 07:30:25.604574 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:25.604541 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" event={"ID":"fb139edb-c8b2-4357-8753-799f76a0dc76","Type":"ContainerStarted","Data":"b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13"} Sep 17 07:30:30.624841 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:30.624736 2566 generic.go:356] "Generic (PLEG): container finished" podID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerID="b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13" exitCode=0 Sep 17 07:30:30.624841 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:30.624818 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" event={"ID":"fb139edb-c8b2-4357-8753-799f76a0dc76","Type":"ContainerDied","Data":"b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13"} Sep 17 07:30:31.635718 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:31.635676 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" event={"ID":"fb139edb-c8b2-4357-8753-799f76a0dc76","Type":"ContainerStarted","Data":"2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d"} Sep 17 07:30:31.636188 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:31.635757 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" event={"ID":"fb139edb-c8b2-4357-8753-799f76a0dc76","Type":"ContainerStarted","Data":"63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b"} Sep 17 07:30:31.636188 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:31.635808 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" event={"ID":"fb139edb-c8b2-4357-8753-799f76a0dc76","Type":"ContainerStarted","Data":"dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc"} Sep 17 07:30:31.636188 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:31.635828 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" event={"ID":"fb139edb-c8b2-4357-8753-799f76a0dc76","Type":"ContainerStarted","Data":"88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4"} Sep 17 07:30:31.665251 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:31.665185 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podStartSLOduration=14.665162848 podStartE2EDuration="14.665162848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:30:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:30:31.662018574 +0000 UTC m=+1344.154093391" watchObservedRunningTime="2026-09-17 07:30:31.665162848 +0000 UTC m=+1344.157237640" Sep 17 07:30:32.350899 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:32.350837 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.19:6567/health\": dial tcp 10.133.0.19:6567: connect: connection refused" Sep 17 07:30:33.809193 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:33.809148 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:33.809676 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:33.809214 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:33.809676 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:33.809235 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:33.809676 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:33.809254 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:33.811524 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:33.811413 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Sep 17 07:30:33.811524 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:33.811445 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.20:8816: connect: connection refused" Sep 17 07:30:33.811524 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:33.811459 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Sep 17 07:30:33.811757 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:33.811544 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.20:6571: connect: connection refused" Sep 17 07:30:35.360829 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:35.360747 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.19:8443: connect: connection refused" Sep 17 07:30:36.809116 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:36.809066 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.20:8816: connect: connection refused" Sep 17 07:30:36.809623 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:36.809131 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Sep 17 07:30:36.809623 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:36.809062 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Sep 17 07:30:36.809623 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:36.809065 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.20:6571: connect: connection refused" Sep 17 07:30:38.808495 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:38.808462 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:38.808871 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:38.808522 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:38.808871 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:38.808539 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:38.808871 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:38.808549 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:39.809662 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:39.809626 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:39.810189 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:39.809965 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:39.810434 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:39.810415 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:39.810507 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:39.810471 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:39.815012 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:39.814990 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:40.677715 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:40.677684 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:40.677897 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:40.677742 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:40.680151 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:40.680140 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:30:42.350656 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:42.350621 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.19:6567/health\": dial tcp 10.133.0.19:6567: connect: connection refused" Sep 17 07:30:42.351087 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:42.350751 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:30:45.360212 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:45.360169 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.19:8443: connect: connection refused" Sep 17 07:30:45.360608 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:45.360292 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:30:47.705827 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:47.705729 2566 generic.go:356] "Generic (PLEG): container finished" podID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerID="4269bc78375557d731dd4509b1b00e96939101935f6392739ff81026db24b046" exitCode=137 Sep 17 07:30:47.705827 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:47.705754 2566 generic.go:356] "Generic (PLEG): container finished" podID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerID="387a2a7e2ed7a3ec921570a71cc3bfff72a90b2c4aa1ab7e5dc2b83400f2e377" exitCode=137 Sep 17 07:30:47.705827 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:47.705796 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" event={"ID":"21c5de96-e2d7-49bd-b490-22f0a853126a","Type":"ContainerDied","Data":"4269bc78375557d731dd4509b1b00e96939101935f6392739ff81026db24b046"} Sep 17 07:30:47.705827 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:47.705826 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" event={"ID":"21c5de96-e2d7-49bd-b490-22f0a853126a","Type":"ContainerDied","Data":"387a2a7e2ed7a3ec921570a71cc3bfff72a90b2c4aa1ab7e5dc2b83400f2e377"} Sep 17 07:30:48.001939 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.001912 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:30:48.100153 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.100123 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/21c5de96-e2d7-49bd-b490-22f0a853126a-feast-data\") pod \"21c5de96-e2d7-49bd-b490-22f0a853126a\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " Sep 17 07:30:48.100346 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.100172 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmqvq\" (UniqueName: \"kubernetes.io/projected/21c5de96-e2d7-49bd-b490-22f0a853126a-kube-api-access-pmqvq\") pod \"21c5de96-e2d7-49bd-b490-22f0a853126a\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " Sep 17 07:30:48.100346 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.100212 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-ui-tls\") pod \"21c5de96-e2d7-49bd-b490-22f0a853126a\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " Sep 17 07:30:48.100346 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.100228 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-registry-tls\") pod \"21c5de96-e2d7-49bd-b490-22f0a853126a\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " Sep 17 07:30:48.100346 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.100256 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-offline-tls\") pod \"21c5de96-e2d7-49bd-b490-22f0a853126a\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " Sep 17 07:30:48.100346 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.100280 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-online-tls\") pod \"21c5de96-e2d7-49bd-b490-22f0a853126a\" (UID: \"21c5de96-e2d7-49bd-b490-22f0a853126a\") " Sep 17 07:30:48.100714 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.100670 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c5de96-e2d7-49bd-b490-22f0a853126a-feast-data" (OuterVolumeSpecName: "feast-data") pod "21c5de96-e2d7-49bd-b490-22f0a853126a" (UID: "21c5de96-e2d7-49bd-b490-22f0a853126a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:30:48.102507 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.102480 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "21c5de96-e2d7-49bd-b490-22f0a853126a" (UID: "21c5de96-e2d7-49bd-b490-22f0a853126a"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:30:48.102824 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.102770 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-online-tls" (OuterVolumeSpecName: "online-tls") pod "21c5de96-e2d7-49bd-b490-22f0a853126a" (UID: "21c5de96-e2d7-49bd-b490-22f0a853126a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:30:48.102940 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.102888 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "21c5de96-e2d7-49bd-b490-22f0a853126a" (UID: "21c5de96-e2d7-49bd-b490-22f0a853126a"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:30:48.102940 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.102891 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21c5de96-e2d7-49bd-b490-22f0a853126a-kube-api-access-pmqvq" (OuterVolumeSpecName: "kube-api-access-pmqvq") pod "21c5de96-e2d7-49bd-b490-22f0a853126a" (UID: "21c5de96-e2d7-49bd-b490-22f0a853126a"). InnerVolumeSpecName "kube-api-access-pmqvq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:30:48.103053 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.102952 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "21c5de96-e2d7-49bd-b490-22f0a853126a" (UID: "21c5de96-e2d7-49bd-b490-22f0a853126a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:30:48.200868 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.200834 2566 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-offline-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:30:48.200868 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.200861 2566 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-online-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:30:48.200868 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.200870 2566 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/21c5de96-e2d7-49bd-b490-22f0a853126a-feast-data\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:30:48.201161 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.200877 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmqvq\" (UniqueName: \"kubernetes.io/projected/21c5de96-e2d7-49bd-b490-22f0a853126a-kube-api-access-pmqvq\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:30:48.201161 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.200886 2566 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-ui-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:30:48.201161 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.200894 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21c5de96-e2d7-49bd-b490-22f0a853126a-registry-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:30:48.712431 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.712400 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" Sep 17 07:30:48.712431 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.712402 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd" event={"ID":"21c5de96-e2d7-49bd-b490-22f0a853126a","Type":"ContainerDied","Data":"83f2a6d63ff9cfcfe983823ce719b6f71486d3d7e38cdc843f3ffa3394a46e6f"} Sep 17 07:30:48.712944 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.712462 2566 scope.go:117] "RemoveContainer" containerID="0802b51d7f2eaf14a9c22ddbc824de13582a490e44adcd196dbf973e288cb69c" Sep 17 07:30:48.720345 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.720268 2566 scope.go:117] "RemoveContainer" containerID="4269bc78375557d731dd4509b1b00e96939101935f6392739ff81026db24b046" Sep 17 07:30:48.727662 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.727646 2566 scope.go:117] "RemoveContainer" containerID="cbe00a71198c1d4e37923058a76cfac409a059fb16f8adc1f4306982ea3613d7" Sep 17 07:30:48.734544 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.734522 2566 scope.go:117] "RemoveContainer" containerID="387a2a7e2ed7a3ec921570a71cc3bfff72a90b2c4aa1ab7e5dc2b83400f2e377" Sep 17 07:30:48.739352 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.739330 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd"] Sep 17 07:30:48.742077 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.742001 2566 scope.go:117] "RemoveContainer" containerID="40c3fcd6576da73c95ea9a99ba3b9b6cbd73f69baba85c2a477c441d38fb589b" Sep 17 07:30:48.743905 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.743882 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-lrlzd"] Sep 17 07:30:48.749195 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:48.749179 2566 scope.go:117] "RemoveContainer" containerID="0f7eb0b215134d7c94e1ea1bcecd9d6d785717b7b6a04634564976ba67d4a785" Sep 17 07:30:50.022957 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:30:50.022926 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" path="/var/lib/kubelet/pods/21c5de96-e2d7-49bd-b490-22f0a853126a/volumes" Sep 17 07:31:42.193148 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.193109 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:42.293695 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.293658 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:42.593845 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.593810 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:42.705071 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.705027 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:42.803677 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.803643 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:42.834179 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834145 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn"] Sep 17 07:31:42.834387 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834376 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="offline" Sep 17 07:31:42.834431 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834389 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="offline" Sep 17 07:31:42.834431 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834398 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="registry" Sep 17 07:31:42.834431 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834403 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="registry" Sep 17 07:31:42.834431 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834413 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="ui" Sep 17 07:31:42.834431 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834418 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="ui" Sep 17 07:31:42.834431 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834427 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="feast-init" Sep 17 07:31:42.834431 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834432 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="feast-init" Sep 17 07:31:42.834657 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834440 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="feast-apply" Sep 17 07:31:42.834657 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834445 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="feast-apply" Sep 17 07:31:42.834657 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834454 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="online" Sep 17 07:31:42.834657 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834459 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="online" Sep 17 07:31:42.834657 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834499 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="registry" Sep 17 07:31:42.834657 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834507 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="ui" Sep 17 07:31:42.834657 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834514 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="online" Sep 17 07:31:42.834657 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.834520 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="21c5de96-e2d7-49bd-b490-22f0a853126a" containerName="offline" Sep 17 07:31:42.837262 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.837241 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:42.839552 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.839520 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 17 07:31:42.840094 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.840077 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 17 07:31:42.840094 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.840085 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 17 07:31:42.840428 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.840408 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-pqfwj\"" Sep 17 07:31:42.840522 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.840501 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 17 07:31:42.840522 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.840514 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 17 07:31:42.840632 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.840515 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 17 07:31:42.851891 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.851867 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn"] Sep 17 07:31:42.880556 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.880519 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:42.902633 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.902592 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:42.902633 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.902633 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:42.902934 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.902686 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9tw6\" (UniqueName: \"kubernetes.io/projected/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-kube-api-access-p9tw6\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:42.902934 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.902716 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:42.902934 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.902773 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:42.902934 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:42.902890 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:43.003328 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.003291 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9tw6\" (UniqueName: \"kubernetes.io/projected/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-kube-api-access-p9tw6\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:43.003328 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.003330 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:43.003573 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.003367 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:43.003573 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.003395 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:43.003573 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.003421 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:43.003573 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.003440 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:43.003973 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.003945 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:43.004210 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.004187 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:43.005985 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.005962 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:43.006087 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.006014 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:43.006167 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.006146 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:43.011711 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.011682 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9tw6\" (UniqueName: \"kubernetes.io/projected/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-kube-api-access-p9tw6\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-s44qn\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:43.025219 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.025183 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:43.146506 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.146412 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:31:43.424156 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.424077 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn"] Sep 17 07:31:43.427683 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:31:43.427659 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29d23db5_5bd8_45ca_8556_2ebcc6c792c9.slice/crio-78c7de3fb53b3a6c5d38a68ddc1a6af1f44704f1303d595aa4fdcd7fc6d8c899 WatchSource:0}: Error finding container 78c7de3fb53b3a6c5d38a68ddc1a6af1f44704f1303d595aa4fdcd7fc6d8c899: Status 404 returned error can't find the container with id 78c7de3fb53b3a6c5d38a68ddc1a6af1f44704f1303d595aa4fdcd7fc6d8c899 Sep 17 07:31:43.912023 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.911986 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" event={"ID":"29d23db5-5bd8-45ca-8556-2ebcc6c792c9","Type":"ContainerStarted","Data":"9af94ff733a064aa3a6fb869666d0a50a96222cfe50a995e16318379e6a4e98d"} Sep 17 07:31:43.912198 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:43.912032 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" event={"ID":"29d23db5-5bd8-45ca-8556-2ebcc6c792c9","Type":"ContainerStarted","Data":"78c7de3fb53b3a6c5d38a68ddc1a6af1f44704f1303d595aa4fdcd7fc6d8c899"} Sep 17 07:31:48.932120 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:48.932088 2566 generic.go:356] "Generic (PLEG): container finished" podID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerID="9af94ff733a064aa3a6fb869666d0a50a96222cfe50a995e16318379e6a4e98d" exitCode=0 Sep 17 07:31:48.932524 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:48.932145 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" event={"ID":"29d23db5-5bd8-45ca-8556-2ebcc6c792c9","Type":"ContainerDied","Data":"9af94ff733a064aa3a6fb869666d0a50a96222cfe50a995e16318379e6a4e98d"} Sep 17 07:31:49.938414 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:31:49.938381 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" event={"ID":"29d23db5-5bd8-45ca-8556-2ebcc6c792c9","Type":"ContainerStarted","Data":"e85c3a2d77c523eba913d4c59c111de0ab5ca53331cd7c56c4c9057cc30f1835"} Sep 17 07:32:15.030566 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:15.030484 2566 generic.go:356] "Generic (PLEG): container finished" podID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerID="e85c3a2d77c523eba913d4c59c111de0ab5ca53331cd7c56c4c9057cc30f1835" exitCode=0 Sep 17 07:32:15.030566 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:15.030534 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" event={"ID":"29d23db5-5bd8-45ca-8556-2ebcc6c792c9","Type":"ContainerDied","Data":"e85c3a2d77c523eba913d4c59c111de0ab5ca53331cd7c56c4c9057cc30f1835"} Sep 17 07:32:16.038461 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:16.038417 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" event={"ID":"29d23db5-5bd8-45ca-8556-2ebcc6c792c9","Type":"ContainerStarted","Data":"4bf1c1c0915ecf61b7a346e87f529e217263215c0c8468030f143ba101112e9a"} Sep 17 07:32:16.038846 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:16.038480 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" event={"ID":"29d23db5-5bd8-45ca-8556-2ebcc6c792c9","Type":"ContainerStarted","Data":"562095086f01b821f3aeb6eedbc5798dd4a35c0f6b1c266f6951014983e3710a"} Sep 17 07:32:16.038846 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:16.038500 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" event={"ID":"29d23db5-5bd8-45ca-8556-2ebcc6c792c9","Type":"ContainerStarted","Data":"d3af7f09001e7011b0ca5f9c427fdcac8c90fb69db69389470f70bb378a821aa"} Sep 17 07:32:16.060414 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:16.060360 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podStartSLOduration=34.06034499 podStartE2EDuration="34.06034499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:31:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:32:16.059373967 +0000 UTC m=+1448.551448790" watchObservedRunningTime="2026-09-17 07:32:16.06034499 +0000 UTC m=+1448.552419783" Sep 17 07:32:16.147577 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:16.147528 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:32:16.147763 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:16.147604 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:32:16.147763 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:16.147624 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:32:16.148943 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:16.148912 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused" Sep 17 07:32:16.149050 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:16.148942 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.21:8816: connect: connection refused" Sep 17 07:32:16.149050 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:16.148984 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused" Sep 17 07:32:16.263441 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:16.263401 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:32:19.147416 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:19.147373 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.21:8816: connect: connection refused" Sep 17 07:32:19.147416 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:19.147401 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused" Sep 17 07:32:19.147977 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:19.147386 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused" Sep 17 07:32:22.147566 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:22.147514 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.21:8816: connect: connection refused" Sep 17 07:32:22.147566 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:22.147557 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused" Sep 17 07:32:22.148257 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:22.147607 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused" Sep 17 07:32:23.147450 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:23.147394 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:32:23.147628 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:23.147463 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:32:23.147628 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:23.147477 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:32:25.148435 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:25.148405 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:32:25.148945 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:25.148579 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:32:25.149147 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:25.149125 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:32:25.149402 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:25.149384 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:32:26.147944 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:26.147895 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Sep 17 07:32:29.151948 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:29.151906 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 17 07:32:32.024701 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:32.024671 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:32:32.908161 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:32.908123 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:32:35.427652 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:35.427619 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:32:44.150941 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:32:44.150894 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 17 07:33:08.022175 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:08.022143 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/1.log" Sep 17 07:33:08.022649 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:08.022593 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/1.log" Sep 17 07:33:08.029201 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:08.029175 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:33:08.029452 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:08.029437 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:33:35.368272 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:35.368195 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn"] Sep 17 07:33:35.368775 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:35.368431 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="online" containerID="cri-o://d3af7f09001e7011b0ca5f9c427fdcac8c90fb69db69389470f70bb378a821aa" gracePeriod=30 Sep 17 07:33:35.368775 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:35.368491 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="offline" containerID="cri-o://562095086f01b821f3aeb6eedbc5798dd4a35c0f6b1c266f6951014983e3710a" gracePeriod=30 Sep 17 07:33:35.368775 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:35.368495 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="ui" containerID="cri-o://4bf1c1c0915ecf61b7a346e87f529e217263215c0c8468030f143ba101112e9a" gracePeriod=30 Sep 17 07:33:35.489429 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:35.489394 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7"] Sep 17 07:33:35.489707 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:35.489644 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="registry" containerID="cri-o://88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4" gracePeriod=30 Sep 17 07:33:35.489871 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:35.489713 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="ui" containerID="cri-o://2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d" gracePeriod=30 Sep 17 07:33:35.489871 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:35.489753 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="online" containerID="cri-o://dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc" gracePeriod=30 Sep 17 07:33:35.489871 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:35.489713 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="offline" containerID="cri-o://63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b" gracePeriod=30 Sep 17 07:33:36.126601 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:36.126551 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:36.199146 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:36.199117 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:36.337654 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:36.337628 2566 generic.go:356] "Generic (PLEG): container finished" podID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerID="2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d" exitCode=0 Sep 17 07:33:36.337767 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:36.337701 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" event={"ID":"fb139edb-c8b2-4357-8753-799f76a0dc76","Type":"ContainerDied","Data":"2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d"} Sep 17 07:33:36.340364 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:36.340342 2566 generic.go:356] "Generic (PLEG): container finished" podID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerID="4bf1c1c0915ecf61b7a346e87f529e217263215c0c8468030f143ba101112e9a" exitCode=0 Sep 17 07:33:36.340364 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:36.340361 2566 generic.go:356] "Generic (PLEG): container finished" podID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerID="d3af7f09001e7011b0ca5f9c427fdcac8c90fb69db69389470f70bb378a821aa" exitCode=0 Sep 17 07:33:36.340485 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:36.340389 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" event={"ID":"29d23db5-5bd8-45ca-8556-2ebcc6c792c9","Type":"ContainerDied","Data":"4bf1c1c0915ecf61b7a346e87f529e217263215c0c8468030f143ba101112e9a"} Sep 17 07:33:36.340485 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:36.340417 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" event={"ID":"29d23db5-5bd8-45ca-8556-2ebcc6c792c9","Type":"ContainerDied","Data":"d3af7f09001e7011b0ca5f9c427fdcac8c90fb69db69389470f70bb378a821aa"} Sep 17 07:33:37.349170 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:37.349140 2566 generic.go:356] "Generic (PLEG): container finished" podID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerID="dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc" exitCode=0 Sep 17 07:33:37.349545 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:37.349214 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" event={"ID":"fb139edb-c8b2-4357-8753-799f76a0dc76","Type":"ContainerDied","Data":"dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc"} Sep 17 07:33:40.671519 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:40.671482 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:40.677602 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:40.677564 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Sep 17 07:33:40.677752 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:40.677564 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Sep 17 07:33:42.025829 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:42.025768 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused" Sep 17 07:33:45.149938 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:45.149897 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused" Sep 17 07:33:50.677960 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:50.677917 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Sep 17 07:33:50.678364 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:50.677912 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Sep 17 07:33:52.025619 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:52.025584 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused" Sep 17 07:33:55.149342 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:33:55.149296 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused" Sep 17 07:34:00.677350 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:00.677304 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Sep 17 07:34:00.677350 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:00.677331 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Sep 17 07:34:00.677832 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:00.677447 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:34:00.677832 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:00.677516 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:34:02.025956 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:02.025921 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused" Sep 17 07:34:02.026323 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:02.026025 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:34:05.149087 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.149051 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused" Sep 17 07:34:05.151390 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.149165 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:34:05.453467 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.453436 2566 generic.go:356] "Generic (PLEG): container finished" podID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerID="562095086f01b821f3aeb6eedbc5798dd4a35c0f6b1c266f6951014983e3710a" exitCode=137 Sep 17 07:34:05.453630 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.453504 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" event={"ID":"29d23db5-5bd8-45ca-8556-2ebcc6c792c9","Type":"ContainerDied","Data":"562095086f01b821f3aeb6eedbc5798dd4a35c0f6b1c266f6951014983e3710a"} Sep 17 07:34:05.581325 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.581300 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:34:05.638708 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.638672 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-ui-tls\") pod \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " Sep 17 07:34:05.638708 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.638714 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-feast-data\") pod \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " Sep 17 07:34:05.638999 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.638735 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-registry-tls\") pod \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " Sep 17 07:34:05.638999 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.638753 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-online-tls\") pod \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " Sep 17 07:34:05.638999 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.638779 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-offline-tls\") pod \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " Sep 17 07:34:05.638999 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.638877 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9tw6\" (UniqueName: \"kubernetes.io/projected/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-kube-api-access-p9tw6\") pod \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\" (UID: \"29d23db5-5bd8-45ca-8556-2ebcc6c792c9\") " Sep 17 07:34:05.639197 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.639152 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "29d23db5-5bd8-45ca-8556-2ebcc6c792c9" (UID: "29d23db5-5bd8-45ca-8556-2ebcc6c792c9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:34:05.639297 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.639272 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-feast-data" (OuterVolumeSpecName: "feast-data") pod "29d23db5-5bd8-45ca-8556-2ebcc6c792c9" (UID: "29d23db5-5bd8-45ca-8556-2ebcc6c792c9"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:34:05.641162 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.641126 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "29d23db5-5bd8-45ca-8556-2ebcc6c792c9" (UID: "29d23db5-5bd8-45ca-8556-2ebcc6c792c9"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:34:05.641266 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.641195 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-online-tls" (OuterVolumeSpecName: "online-tls") pod "29d23db5-5bd8-45ca-8556-2ebcc6c792c9" (UID: "29d23db5-5bd8-45ca-8556-2ebcc6c792c9"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:34:05.641266 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.641211 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-kube-api-access-p9tw6" (OuterVolumeSpecName: "kube-api-access-p9tw6") pod "29d23db5-5bd8-45ca-8556-2ebcc6c792c9" (UID: "29d23db5-5bd8-45ca-8556-2ebcc6c792c9"). InnerVolumeSpecName "kube-api-access-p9tw6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:34:05.641266 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.641222 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "29d23db5-5bd8-45ca-8556-2ebcc6c792c9" (UID: "29d23db5-5bd8-45ca-8556-2ebcc6c792c9"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:34:05.739574 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.739480 2566 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-ui-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:34:05.739574 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.739511 2566 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-feast-data\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:34:05.739574 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.739522 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-registry-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:34:05.739574 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.739530 2566 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-online-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:34:05.739574 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.739538 2566 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-offline-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:34:05.739574 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:05.739547 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9tw6\" (UniqueName: \"kubernetes.io/projected/29d23db5-5bd8-45ca-8556-2ebcc6c792c9-kube-api-access-p9tw6\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:34:06.141814 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.141769 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:34:06.242507 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.242469 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-online-tls\") pod \"fb139edb-c8b2-4357-8753-799f76a0dc76\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " Sep 17 07:34:06.243007 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.242522 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fb139edb-c8b2-4357-8753-799f76a0dc76-feast-data\") pod \"fb139edb-c8b2-4357-8753-799f76a0dc76\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " Sep 17 07:34:06.243007 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.242571 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-offline-tls\") pod \"fb139edb-c8b2-4357-8753-799f76a0dc76\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " Sep 17 07:34:06.243007 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.242594 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-ui-tls\") pod \"fb139edb-c8b2-4357-8753-799f76a0dc76\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " Sep 17 07:34:06.243007 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.242635 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-registry-tls\") pod \"fb139edb-c8b2-4357-8753-799f76a0dc76\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " Sep 17 07:34:06.243007 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.242683 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrd9p\" (UniqueName: \"kubernetes.io/projected/fb139edb-c8b2-4357-8753-799f76a0dc76-kube-api-access-nrd9p\") pod \"fb139edb-c8b2-4357-8753-799f76a0dc76\" (UID: \"fb139edb-c8b2-4357-8753-799f76a0dc76\") " Sep 17 07:34:06.243280 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.243210 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb139edb-c8b2-4357-8753-799f76a0dc76-feast-data" (OuterVolumeSpecName: "feast-data") pod "fb139edb-c8b2-4357-8753-799f76a0dc76" (UID: "fb139edb-c8b2-4357-8753-799f76a0dc76"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:34:06.244718 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.244692 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-online-tls" (OuterVolumeSpecName: "online-tls") pod "fb139edb-c8b2-4357-8753-799f76a0dc76" (UID: "fb139edb-c8b2-4357-8753-799f76a0dc76"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:34:06.245162 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.245145 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "fb139edb-c8b2-4357-8753-799f76a0dc76" (UID: "fb139edb-c8b2-4357-8753-799f76a0dc76"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:34:06.245268 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.245217 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fb139edb-c8b2-4357-8753-799f76a0dc76" (UID: "fb139edb-c8b2-4357-8753-799f76a0dc76"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:34:06.245268 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.245228 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb139edb-c8b2-4357-8753-799f76a0dc76-kube-api-access-nrd9p" (OuterVolumeSpecName: "kube-api-access-nrd9p") pod "fb139edb-c8b2-4357-8753-799f76a0dc76" (UID: "fb139edb-c8b2-4357-8753-799f76a0dc76"). InnerVolumeSpecName "kube-api-access-nrd9p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:34:06.245268 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.245236 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "fb139edb-c8b2-4357-8753-799f76a0dc76" (UID: "fb139edb-c8b2-4357-8753-799f76a0dc76"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:34:06.344184 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.344135 2566 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-online-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:34:06.344184 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.344178 2566 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fb139edb-c8b2-4357-8753-799f76a0dc76-feast-data\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:34:06.344184 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.344187 2566 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-offline-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:34:06.344184 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.344195 2566 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-ui-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:34:06.344184 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.344203 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fb139edb-c8b2-4357-8753-799f76a0dc76-registry-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:34:06.344553 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.344211 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrd9p\" (UniqueName: \"kubernetes.io/projected/fb139edb-c8b2-4357-8753-799f76a0dc76-kube-api-access-nrd9p\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:34:06.459301 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.459261 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" event={"ID":"29d23db5-5bd8-45ca-8556-2ebcc6c792c9","Type":"ContainerDied","Data":"78c7de3fb53b3a6c5d38a68ddc1a6af1f44704f1303d595aa4fdcd7fc6d8c899"} Sep 17 07:34:06.459477 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.459320 2566 scope.go:117] "RemoveContainer" containerID="4bf1c1c0915ecf61b7a346e87f529e217263215c0c8468030f143ba101112e9a" Sep 17 07:34:06.459477 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.459277 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn" Sep 17 07:34:06.462703 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.462678 2566 generic.go:356] "Generic (PLEG): container finished" podID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerID="63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b" exitCode=137 Sep 17 07:34:06.462703 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.462699 2566 generic.go:356] "Generic (PLEG): container finished" podID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerID="88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4" exitCode=137 Sep 17 07:34:06.462880 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.462731 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" event={"ID":"fb139edb-c8b2-4357-8753-799f76a0dc76","Type":"ContainerDied","Data":"63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b"} Sep 17 07:34:06.462880 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.462737 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" Sep 17 07:34:06.462880 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.462758 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" event={"ID":"fb139edb-c8b2-4357-8753-799f76a0dc76","Type":"ContainerDied","Data":"88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4"} Sep 17 07:34:06.462880 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.462776 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7" event={"ID":"fb139edb-c8b2-4357-8753-799f76a0dc76","Type":"ContainerDied","Data":"1009994a3a8d22e44011b34c0749180205dc4610a96f9931eb2b4f278c9e496d"} Sep 17 07:34:06.467747 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.467730 2566 scope.go:117] "RemoveContainer" containerID="562095086f01b821f3aeb6eedbc5798dd4a35c0f6b1c266f6951014983e3710a" Sep 17 07:34:06.475586 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.475566 2566 scope.go:117] "RemoveContainer" containerID="d3af7f09001e7011b0ca5f9c427fdcac8c90fb69db69389470f70bb378a821aa" Sep 17 07:34:06.476956 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.476932 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn"] Sep 17 07:34:06.482847 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.482820 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-s44qn"] Sep 17 07:34:06.484710 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.484692 2566 scope.go:117] "RemoveContainer" containerID="e85c3a2d77c523eba913d4c59c111de0ab5ca53331cd7c56c4c9057cc30f1835" Sep 17 07:34:06.492208 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.492186 2566 scope.go:117] "RemoveContainer" containerID="9af94ff733a064aa3a6fb869666d0a50a96222cfe50a995e16318379e6a4e98d" Sep 17 07:34:06.499546 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.499525 2566 scope.go:117] "RemoveContainer" containerID="2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d" Sep 17 07:34:06.506327 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.506299 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7"] Sep 17 07:34:06.507322 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.507307 2566 scope.go:117] "RemoveContainer" containerID="63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b" Sep 17 07:34:06.510129 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.510109 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-sjtn7"] Sep 17 07:34:06.514973 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.514958 2566 scope.go:117] "RemoveContainer" containerID="dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc" Sep 17 07:34:06.521987 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.521967 2566 scope.go:117] "RemoveContainer" containerID="88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4" Sep 17 07:34:06.528668 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.528650 2566 scope.go:117] "RemoveContainer" containerID="b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13" Sep 17 07:34:06.535484 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.535463 2566 scope.go:117] "RemoveContainer" containerID="cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191" Sep 17 07:34:06.542326 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.542301 2566 scope.go:117] "RemoveContainer" containerID="2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d" Sep 17 07:34:06.542638 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:34:06.542613 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d\": container with ID starting with 2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d not found: ID does not exist" containerID="2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d" Sep 17 07:34:06.542696 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.542643 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d"} err="failed to get container status \"2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d\": rpc error: code = NotFound desc = could not find container \"2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d\": container with ID starting with 2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d not found: ID does not exist" Sep 17 07:34:06.542696 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.542663 2566 scope.go:117] "RemoveContainer" containerID="63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b" Sep 17 07:34:06.542964 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:34:06.542942 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b\": container with ID starting with 63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b not found: ID does not exist" containerID="63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b" Sep 17 07:34:06.543057 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.542970 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b"} err="failed to get container status \"63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b\": rpc error: code = NotFound desc = could not find container \"63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b\": container with ID starting with 63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b not found: ID does not exist" Sep 17 07:34:06.543057 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.542986 2566 scope.go:117] "RemoveContainer" containerID="dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc" Sep 17 07:34:06.543250 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:34:06.543236 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc\": container with ID starting with dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc not found: ID does not exist" containerID="dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc" Sep 17 07:34:06.543290 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.543253 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc"} err="failed to get container status \"dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc\": rpc error: code = NotFound desc = could not find container \"dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc\": container with ID starting with dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc not found: ID does not exist" Sep 17 07:34:06.543290 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.543273 2566 scope.go:117] "RemoveContainer" containerID="88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4" Sep 17 07:34:06.543515 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:34:06.543500 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4\": container with ID starting with 88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4 not found: ID does not exist" containerID="88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4" Sep 17 07:34:06.543574 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.543517 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4"} err="failed to get container status \"88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4\": rpc error: code = NotFound desc = could not find container \"88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4\": container with ID starting with 88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4 not found: ID does not exist" Sep 17 07:34:06.543574 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.543529 2566 scope.go:117] "RemoveContainer" containerID="b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13" Sep 17 07:34:06.543777 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:34:06.543758 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13\": container with ID starting with b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13 not found: ID does not exist" containerID="b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13" Sep 17 07:34:06.543876 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.543805 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13"} err="failed to get container status \"b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13\": rpc error: code = NotFound desc = could not find container \"b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13\": container with ID starting with b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13 not found: ID does not exist" Sep 17 07:34:06.543876 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.543823 2566 scope.go:117] "RemoveContainer" containerID="cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191" Sep 17 07:34:06.544073 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:34:06.544054 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191\": container with ID starting with cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191 not found: ID does not exist" containerID="cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191" Sep 17 07:34:06.544118 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.544077 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191"} err="failed to get container status \"cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191\": rpc error: code = NotFound desc = could not find container \"cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191\": container with ID starting with cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191 not found: ID does not exist" Sep 17 07:34:06.544118 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.544089 2566 scope.go:117] "RemoveContainer" containerID="2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d" Sep 17 07:34:06.544310 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.544294 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d"} err="failed to get container status \"2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d\": rpc error: code = NotFound desc = could not find container \"2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d\": container with ID starting with 2973354f8061e0325d1982f58ca6c87c2c33f97ecc396ce46ea6d9065ffeeb8d not found: ID does not exist" Sep 17 07:34:06.544348 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.544310 2566 scope.go:117] "RemoveContainer" containerID="63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b" Sep 17 07:34:06.544510 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.544496 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b"} err="failed to get container status \"63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b\": rpc error: code = NotFound desc = could not find container \"63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b\": container with ID starting with 63b2b970f07e1a49c7d8399b9e4e365459aa0f48f98b4376d71e8b4985548e1b not found: ID does not exist" Sep 17 07:34:06.544510 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.544510 2566 scope.go:117] "RemoveContainer" containerID="dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc" Sep 17 07:34:06.544713 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.544698 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc"} err="failed to get container status \"dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc\": rpc error: code = NotFound desc = could not find container \"dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc\": container with ID starting with dcc696d591e5857f8baf8e6499d7d6cae41058a152414c2a7d28fa26c33972cc not found: ID does not exist" Sep 17 07:34:06.544754 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.544714 2566 scope.go:117] "RemoveContainer" containerID="88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4" Sep 17 07:34:06.544935 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.544920 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4"} err="failed to get container status \"88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4\": rpc error: code = NotFound desc = could not find container \"88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4\": container with ID starting with 88a09e90e9fbc66e27da7efd4953256304821e2a0e9af66e8a0e9b395ea99ea4 not found: ID does not exist" Sep 17 07:34:06.544994 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.544935 2566 scope.go:117] "RemoveContainer" containerID="b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13" Sep 17 07:34:06.545167 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.545149 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13"} err="failed to get container status \"b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13\": rpc error: code = NotFound desc = could not find container \"b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13\": container with ID starting with b49e27e69a13edd88af01ea8b44847ff58500e39da1f85f96eb4efd8b74dfd13 not found: ID does not exist" Sep 17 07:34:06.545215 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.545168 2566 scope.go:117] "RemoveContainer" containerID="cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191" Sep 17 07:34:06.545408 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:06.545391 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191"} err="failed to get container status \"cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191\": rpc error: code = NotFound desc = could not find container \"cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191\": container with ID starting with cc81e923fde25c948040abdd87a69b627f9dec7e051234e50737767fda89b191 not found: ID does not exist" Sep 17 07:34:08.022458 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:08.022428 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" path="/var/lib/kubelet/pods/29d23db5-5bd8-45ca-8556-2ebcc6c792c9/volumes" Sep 17 07:34:08.023002 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:08.022987 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" path="/var/lib/kubelet/pods/fb139edb-c8b2-4357-8753-799f76a0dc76/volumes" Sep 17 07:34:18.337377 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:18.337337 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:18.451578 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:18.451535 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:24.257878 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:24.257834 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:24.315333 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:24.315293 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:24.370478 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:24.370438 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t"] Sep 17 07:34:24.370672 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:24.370648 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t" podUID="b338bf76-721e-4a76-9287-a1c4f5aaf95d" containerName="manager" containerID="cri-o://83598caa78a217ec59e6c19d1a2707bd7e0aead64b42283b149f4478833ae21e" gracePeriod=10 Sep 17 07:34:24.526666 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:24.526578 2566 generic.go:356] "Generic (PLEG): container finished" podID="b338bf76-721e-4a76-9287-a1c4f5aaf95d" containerID="83598caa78a217ec59e6c19d1a2707bd7e0aead64b42283b149f4478833ae21e" exitCode=0 Sep 17 07:34:24.526813 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:24.526653 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t" event={"ID":"b338bf76-721e-4a76-9287-a1c4f5aaf95d","Type":"ContainerDied","Data":"83598caa78a217ec59e6c19d1a2707bd7e0aead64b42283b149f4478833ae21e"} Sep 17 07:34:24.608326 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:24.608298 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t" Sep 17 07:34:24.777453 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:24.777365 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vw8w7\" (UniqueName: \"kubernetes.io/projected/b338bf76-721e-4a76-9287-a1c4f5aaf95d-kube-api-access-vw8w7\") pod \"b338bf76-721e-4a76-9287-a1c4f5aaf95d\" (UID: \"b338bf76-721e-4a76-9287-a1c4f5aaf95d\") " Sep 17 07:34:24.779618 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:24.779591 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b338bf76-721e-4a76-9287-a1c4f5aaf95d-kube-api-access-vw8w7" (OuterVolumeSpecName: "kube-api-access-vw8w7") pod "b338bf76-721e-4a76-9287-a1c4f5aaf95d" (UID: "b338bf76-721e-4a76-9287-a1c4f5aaf95d"). InnerVolumeSpecName "kube-api-access-vw8w7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:34:24.877881 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:24.877843 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vw8w7\" (UniqueName: \"kubernetes.io/projected/b338bf76-721e-4a76-9287-a1c4f5aaf95d-kube-api-access-vw8w7\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:34:25.515886 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:25.515847 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:25.531516 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:25.531488 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t" Sep 17 07:34:25.531673 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:25.531487 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t" event={"ID":"b338bf76-721e-4a76-9287-a1c4f5aaf95d","Type":"ContainerDied","Data":"a028f1b270574043e82750f2fe8eb42a245c4ad9985f65c3641c577b1154139d"} Sep 17 07:34:25.531673 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:25.531627 2566 scope.go:117] "RemoveContainer" containerID="83598caa78a217ec59e6c19d1a2707bd7e0aead64b42283b149f4478833ae21e" Sep 17 07:34:25.554507 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:25.554480 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t"] Sep 17 07:34:25.558670 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:25.558646 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-5pr6t"] Sep 17 07:34:26.021837 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:26.021777 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b338bf76-721e-4a76-9287-a1c4f5aaf95d" path="/var/lib/kubelet/pods/b338bf76-721e-4a76-9287-a1c4f5aaf95d/volumes" Sep 17 07:34:29.231258 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231220 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-ccdt2"] Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231448 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="online" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231459 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="online" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231468 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b338bf76-721e-4a76-9287-a1c4f5aaf95d" containerName="manager" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231473 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b338bf76-721e-4a76-9287-a1c4f5aaf95d" containerName="manager" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231481 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="online" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231487 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="online" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231495 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="feast-init" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231501 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="feast-init" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231506 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="offline" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231511 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="offline" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231517 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="offline" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231522 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="offline" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231529 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="feast-init" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231534 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="feast-init" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231543 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="ui" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231548 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="ui" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231556 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="ui" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231561 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="ui" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231567 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="feast-apply" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231571 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="feast-apply" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231577 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="registry" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231581 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="registry" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231587 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="feast-apply" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231592 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="feast-apply" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231629 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="offline" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231636 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b338bf76-721e-4a76-9287-a1c4f5aaf95d" containerName="manager" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231642 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="ui" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231646 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="offline" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231651 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="registry" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231656 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="29d23db5-5bd8-45ca-8556-2ebcc6c792c9" containerName="online" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231661 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="online" Sep 17 07:34:29.231669 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.231666 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb139edb-c8b2-4357-8753-799f76a0dc76" containerName="ui" Sep 17 07:34:29.235621 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.235604 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-ccdt2" Sep 17 07:34:29.238376 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.238356 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 17 07:34:29.239156 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.239139 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-jhv5r\"" Sep 17 07:34:29.239231 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.239182 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 17 07:34:29.244046 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.244012 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-ccdt2"] Sep 17 07:34:29.414117 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.414074 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5kqz\" (UniqueName: \"kubernetes.io/projected/9eecbd83-ed8b-4e58-9b39-6b9f20de1cfa-kube-api-access-q5kqz\") pod \"feast-operator-controller-manager-689b877b4b-ccdt2\" (UID: \"9eecbd83-ed8b-4e58-9b39-6b9f20de1cfa\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-ccdt2" Sep 17 07:34:29.515487 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.515390 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5kqz\" (UniqueName: \"kubernetes.io/projected/9eecbd83-ed8b-4e58-9b39-6b9f20de1cfa-kube-api-access-q5kqz\") pod \"feast-operator-controller-manager-689b877b4b-ccdt2\" (UID: \"9eecbd83-ed8b-4e58-9b39-6b9f20de1cfa\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-ccdt2" Sep 17 07:34:29.524221 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.524182 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5kqz\" (UniqueName: \"kubernetes.io/projected/9eecbd83-ed8b-4e58-9b39-6b9f20de1cfa-kube-api-access-q5kqz\") pod \"feast-operator-controller-manager-689b877b4b-ccdt2\" (UID: \"9eecbd83-ed8b-4e58-9b39-6b9f20de1cfa\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-ccdt2" Sep 17 07:34:29.546119 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.546094 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-ccdt2" Sep 17 07:34:29.823760 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:29.823716 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-ccdt2"] Sep 17 07:34:29.826762 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:34:29.826733 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eecbd83_ed8b_4e58_9b39_6b9f20de1cfa.slice/crio-d1ba3f7924a916aa1db9cc135b2339caf1876a6e96a08157e1456b597e30ad3e WatchSource:0}: Error finding container d1ba3f7924a916aa1db9cc135b2339caf1876a6e96a08157e1456b597e30ad3e: Status 404 returned error can't find the container with id d1ba3f7924a916aa1db9cc135b2339caf1876a6e96a08157e1456b597e30ad3e Sep 17 07:34:30.550684 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:30.550641 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-ccdt2" event={"ID":"9eecbd83-ed8b-4e58-9b39-6b9f20de1cfa","Type":"ContainerStarted","Data":"935e0da3c208cc582311cb19c17c88c083ffac4a4c5761307eed19722646eab8"} Sep 17 07:34:30.550684 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:30.550691 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-ccdt2" event={"ID":"9eecbd83-ed8b-4e58-9b39-6b9f20de1cfa","Type":"ContainerStarted","Data":"d1ba3f7924a916aa1db9cc135b2339caf1876a6e96a08157e1456b597e30ad3e"} Sep 17 07:34:30.551131 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:30.550722 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-ccdt2" Sep 17 07:34:30.569138 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:30.569089 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-ccdt2" podStartSLOduration=1.569072832 podStartE2EDuration="1.569072832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:34:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:34:30.567745812 +0000 UTC m=+1583.059820605" watchObservedRunningTime="2026-09-17 07:34:30.569072832 +0000 UTC m=+1583.061147625" Sep 17 07:34:36.546247 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:36.546210 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:41.557018 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:41.556988 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-ccdt2" Sep 17 07:34:45.639566 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:45.639527 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:45.741210 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:45.741172 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:47.399949 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.399913 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:47.501292 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.501257 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:47.552591 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.552553 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm"] Sep 17 07:34:47.556004 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.555980 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.558689 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.558668 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 17 07:34:47.558994 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.558980 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-hzmw5\"" Sep 17 07:34:47.559886 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.559727 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 17 07:34:47.559886 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.559821 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 17 07:34:47.560053 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.560003 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 07:34:47.563798 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.560223 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 07:34:47.563798 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.560365 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 17 07:34:47.570710 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.570685 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm"] Sep 17 07:34:47.606163 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.606127 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:47.648506 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.648466 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5287\" (UniqueName: \"kubernetes.io/projected/4dab69bc-e18b-4031-8ea7-823b53775599-kube-api-access-b5287\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.648506 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.648504 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.648724 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.648526 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.648724 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.648570 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.648724 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.648593 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4dab69bc-e18b-4031-8ea7-823b53775599-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.648724 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.648613 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.700670 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.700565 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:47.749593 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.749564 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5287\" (UniqueName: \"kubernetes.io/projected/4dab69bc-e18b-4031-8ea7-823b53775599-kube-api-access-b5287\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.749769 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.749599 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.749769 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.749622 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.749769 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.749644 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.749769 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.749668 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4dab69bc-e18b-4031-8ea7-823b53775599-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.749769 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.749695 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.750212 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.750169 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4dab69bc-e18b-4031-8ea7-823b53775599-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.752406 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.752381 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.752498 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.752385 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.752498 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.752421 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.752575 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.752494 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.756944 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.756920 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5287\" (UniqueName: \"kubernetes.io/projected/4dab69bc-e18b-4031-8ea7-823b53775599-kube-api-access-b5287\") pod \"feast-simple-feast-setup-57576b8fcf-8wpfm\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:47.801503 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.801459 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:47.868855 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:47.868818 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:34:48.144535 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:48.144485 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm"] Sep 17 07:34:48.147554 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:34:48.147525 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4dab69bc_e18b_4031_8ea7_823b53775599.slice/crio-9c197e27935a75a7f8e88b840d8dd2d9f95609690919bab4b393e5f8e43d1d5d WatchSource:0}: Error finding container 9c197e27935a75a7f8e88b840d8dd2d9f95609690919bab4b393e5f8e43d1d5d: Status 404 returned error can't find the container with id 9c197e27935a75a7f8e88b840d8dd2d9f95609690919bab4b393e5f8e43d1d5d Sep 17 07:34:48.617272 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:48.617235 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" event={"ID":"4dab69bc-e18b-4031-8ea7-823b53775599","Type":"ContainerStarted","Data":"4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b"} Sep 17 07:34:48.617709 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:48.617283 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" event={"ID":"4dab69bc-e18b-4031-8ea7-823b53775599","Type":"ContainerStarted","Data":"9c197e27935a75a7f8e88b840d8dd2d9f95609690919bab4b393e5f8e43d1d5d"} Sep 17 07:34:53.637891 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:53.637853 2566 generic.go:356] "Generic (PLEG): container finished" podID="4dab69bc-e18b-4031-8ea7-823b53775599" containerID="4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b" exitCode=0 Sep 17 07:34:53.638267 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:53.637929 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" event={"ID":"4dab69bc-e18b-4031-8ea7-823b53775599","Type":"ContainerDied","Data":"4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b"} Sep 17 07:34:54.644746 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:34:54.644712 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" event={"ID":"4dab69bc-e18b-4031-8ea7-823b53775599","Type":"ContainerStarted","Data":"c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae"} Sep 17 07:35:00.667454 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:00.667379 2566 generic.go:356] "Generic (PLEG): container finished" podID="4dab69bc-e18b-4031-8ea7-823b53775599" containerID="c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae" exitCode=0 Sep 17 07:35:00.667819 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:00.667451 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" event={"ID":"4dab69bc-e18b-4031-8ea7-823b53775599","Type":"ContainerDied","Data":"c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae"} Sep 17 07:35:01.677455 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:01.677412 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" event={"ID":"4dab69bc-e18b-4031-8ea7-823b53775599","Type":"ContainerStarted","Data":"6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97"} Sep 17 07:35:01.677966 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:01.677480 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" event={"ID":"4dab69bc-e18b-4031-8ea7-823b53775599","Type":"ContainerStarted","Data":"53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40"} Sep 17 07:35:01.677966 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:01.677501 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" event={"ID":"4dab69bc-e18b-4031-8ea7-823b53775599","Type":"ContainerStarted","Data":"1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc"} Sep 17 07:35:01.677966 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:01.677520 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" event={"ID":"4dab69bc-e18b-4031-8ea7-823b53775599","Type":"ContainerStarted","Data":"eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1"} Sep 17 07:35:01.709163 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:01.709096 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podStartSLOduration=14.709071633 podStartE2EDuration="14.709071633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:34:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:35:01.705832942 +0000 UTC m=+1614.197907736" watchObservedRunningTime="2026-09-17 07:35:01.709071633 +0000 UTC m=+1614.201146427" Sep 17 07:35:02.870760 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:02.869830 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:35:02.870760 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:02.869899 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:35:02.870760 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:02.869920 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:35:02.870760 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:02.870365 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:35:02.871889 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:02.871859 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.23:8816: connect: connection refused" Sep 17 07:35:02.872120 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:02.871890 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Sep 17 07:35:02.872198 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:02.871904 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.23:6571: connect: connection refused" Sep 17 07:35:02.872198 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:02.871948 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Sep 17 07:35:05.869976 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:05.869929 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.23:8816: connect: connection refused" Sep 17 07:35:05.870599 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:05.869937 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Sep 17 07:35:05.870800 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:05.869930 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Sep 17 07:35:05.870939 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:05.869990 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.23:6571: connect: connection refused" Sep 17 07:35:07.869140 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:07.869097 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:35:07.869756 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:07.869719 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:35:07.869756 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:07.869756 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:35:07.869975 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:07.869797 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:35:08.869736 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:08.869690 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Sep 17 07:35:08.870701 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:08.870678 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:35:08.870821 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:08.870774 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:35:08.876045 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:08.876022 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:35:09.715942 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:09.715906 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:35:09.716124 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:09.715984 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:35:09.718469 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:09.718448 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:35:11.870691 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:11.870656 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:35:12.726552 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:35:12.726523 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:36:14.369160 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:14.369124 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:36:14.370860 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:14.370831 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm"] Sep 17 07:36:14.371178 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:14.371151 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="registry" containerID="cri-o://eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1" gracePeriod=30 Sep 17 07:36:14.371257 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:14.371156 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="ui" containerID="cri-o://6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97" gracePeriod=30 Sep 17 07:36:14.371257 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:14.371181 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="online" containerID="cri-o://1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc" gracePeriod=30 Sep 17 07:36:14.371257 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:14.371156 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="offline" containerID="cri-o://53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40" gracePeriod=30 Sep 17 07:36:14.876742 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:14.876708 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5"] Sep 17 07:36:14.880299 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:14.880273 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:14.882521 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:14.882488 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-flwhl\"" Sep 17 07:36:14.894552 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:14.894519 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5"] Sep 17 07:36:14.970287 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:14.970252 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:14.970471 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:14.970313 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:14.970471 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:14.970348 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/87909099-ca75-46bf-93b3-de2c9ca4e102-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:14.970471 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:14.970440 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:14.970471 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:14.970460 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:14.970629 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:14.970488 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlmjj\" (UniqueName: \"kubernetes.io/projected/87909099-ca75-46bf-93b3-de2c9ca4e102-kube-api-access-hlmjj\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:15.071996 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.071945 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/87909099-ca75-46bf-93b3-de2c9ca4e102-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:15.072186 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.072134 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:15.072246 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.072182 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:15.072246 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.072217 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlmjj\" (UniqueName: \"kubernetes.io/projected/87909099-ca75-46bf-93b3-de2c9ca4e102-kube-api-access-hlmjj\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:15.072330 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.072292 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:15.072383 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.072340 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:15.072418 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.072406 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/87909099-ca75-46bf-93b3-de2c9ca4e102-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:15.072601 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:36:15.072570 2566 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 17 07:36:15.072726 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:36:15.072645 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-ui-tls podName:87909099-ca75-46bf-93b3-de2c9ca4e102 nodeName:}" failed. No retries permitted until 2026-09-17 07:36:15.572627901 +0000 UTC m=+1688.064702672 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-9j2m5" (UID: "87909099-ca75-46bf-93b3-de2c9ca4e102") : secret "feast-simple-feast-setup-ui-tls" not found Sep 17 07:36:15.074864 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.074841 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:15.075294 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.075274 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:15.075445 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.075420 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:15.081036 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.080976 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlmjj\" (UniqueName: \"kubernetes.io/projected/87909099-ca75-46bf-93b3-de2c9ca4e102-kube-api-access-hlmjj\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:15.554292 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.554259 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:36:15.576349 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.576316 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:15.578683 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.578660 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9j2m5\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:15.789403 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.789361 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:15.946481 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.946450 2566 generic.go:356] "Generic (PLEG): container finished" podID="4dab69bc-e18b-4031-8ea7-823b53775599" containerID="6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97" exitCode=0 Sep 17 07:36:15.946481 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.946476 2566 generic.go:356] "Generic (PLEG): container finished" podID="4dab69bc-e18b-4031-8ea7-823b53775599" containerID="1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc" exitCode=0 Sep 17 07:36:15.946690 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.946513 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" event={"ID":"4dab69bc-e18b-4031-8ea7-823b53775599","Type":"ContainerDied","Data":"6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97"} Sep 17 07:36:15.946690 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:15.946547 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" event={"ID":"4dab69bc-e18b-4031-8ea7-823b53775599","Type":"ContainerDied","Data":"1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc"} Sep 17 07:36:16.064531 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:16.064505 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5"] Sep 17 07:36:16.066761 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:36:16.066732 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87909099_ca75_46bf_93b3_de2c9ca4e102.slice/crio-3b274c04d2f4c964d624c17aa5d9bfd4ec1d3fc967ded9d4ed0ccba314e93b24 WatchSource:0}: Error finding container 3b274c04d2f4c964d624c17aa5d9bfd4ec1d3fc967ded9d4ed0ccba314e93b24: Status 404 returned error can't find the container with id 3b274c04d2f4c964d624c17aa5d9bfd4ec1d3fc967ded9d4ed0ccba314e93b24 Sep 17 07:36:16.951853 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:16.951817 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" event={"ID":"87909099-ca75-46bf-93b3-de2c9ca4e102","Type":"ContainerStarted","Data":"1a3a7e08960a413afa52c26cc51a367d0b0c1cc826893cd2c07129eb807148e1"} Sep 17 07:36:16.952242 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:16.951863 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" event={"ID":"87909099-ca75-46bf-93b3-de2c9ca4e102","Type":"ContainerStarted","Data":"3b274c04d2f4c964d624c17aa5d9bfd4ec1d3fc967ded9d4ed0ccba314e93b24"} Sep 17 07:36:19.716108 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:19.716052 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Sep 17 07:36:21.973213 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:21.973179 2566 generic.go:356] "Generic (PLEG): container finished" podID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerID="1a3a7e08960a413afa52c26cc51a367d0b0c1cc826893cd2c07129eb807148e1" exitCode=0 Sep 17 07:36:21.973604 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:21.973257 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" event={"ID":"87909099-ca75-46bf-93b3-de2c9ca4e102","Type":"ContainerDied","Data":"1a3a7e08960a413afa52c26cc51a367d0b0c1cc826893cd2c07129eb807148e1"} Sep 17 07:36:22.726556 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:22.726515 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Sep 17 07:36:22.979579 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:22.979499 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" event={"ID":"87909099-ca75-46bf-93b3-de2c9ca4e102","Type":"ContainerStarted","Data":"69fa8d411db6433f80fc39835e4d6cdfc635380ac09dbf84efd52b7956dea70d"} Sep 17 07:36:29.003294 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:29.003260 2566 generic.go:356] "Generic (PLEG): container finished" podID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerID="69fa8d411db6433f80fc39835e4d6cdfc635380ac09dbf84efd52b7956dea70d" exitCode=0 Sep 17 07:36:29.003706 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:29.003327 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" event={"ID":"87909099-ca75-46bf-93b3-de2c9ca4e102","Type":"ContainerDied","Data":"69fa8d411db6433f80fc39835e4d6cdfc635380ac09dbf84efd52b7956dea70d"} Sep 17 07:36:29.717031 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:29.716971 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Sep 17 07:36:30.014218 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:30.014121 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" event={"ID":"87909099-ca75-46bf-93b3-de2c9ca4e102","Type":"ContainerStarted","Data":"e4089ea2afd1d79608e5dcb718267fd4048f53a18801696aa1dd673d2d2511e9"} Sep 17 07:36:30.014218 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:30.014178 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" event={"ID":"87909099-ca75-46bf-93b3-de2c9ca4e102","Type":"ContainerStarted","Data":"a2b5ae0dd23d3ad8631acfb3024cc2a14c2501105f376e810bab15b17b7d9a5c"} Sep 17 07:36:30.014218 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:30.014197 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" event={"ID":"87909099-ca75-46bf-93b3-de2c9ca4e102","Type":"ContainerStarted","Data":"6c16a72a5625ed21f499d3de9e3b3a271f2e1d2394a08fc364606b62e3f2d7be"} Sep 17 07:36:30.014218 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:30.014214 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" event={"ID":"87909099-ca75-46bf-93b3-de2c9ca4e102","Type":"ContainerStarted","Data":"465fe87fd7e9691059a1924720554ffd9af689356668a47ca4c8c7a73eb9ff20"} Sep 17 07:36:30.044337 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:30.044269 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podStartSLOduration=16.044242846 podStartE2EDuration="16.044242846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:36:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:36:30.042283422 +0000 UTC m=+1702.534358244" watchObservedRunningTime="2026-09-17 07:36:30.044242846 +0000 UTC m=+1702.536317640" Sep 17 07:36:30.789778 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:30.789674 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:30.789778 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:30.789744 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:30.789778 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:30.789764 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:30.789778 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:30.789798 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:30.791964 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:30.791912 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 17 07:36:30.791964 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:30.791937 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.24:8816: connect: connection refused" Sep 17 07:36:30.792174 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:30.791961 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Sep 17 07:36:30.792174 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:30.792072 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.24:6571: connect: connection refused" Sep 17 07:36:32.727086 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:32.727032 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Sep 17 07:36:33.790414 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:33.790356 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 17 07:36:33.790944 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:33.790367 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Sep 17 07:36:33.790944 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:33.790372 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.24:8816: connect: connection refused" Sep 17 07:36:33.790944 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:33.790901 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.24:6571: connect: connection refused" Sep 17 07:36:35.790363 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:35.790306 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:35.790890 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:35.790388 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:35.790890 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:35.790411 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:35.790890 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:35.790428 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:36.790728 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:36.790545 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.24:6571: connect: connection refused" Sep 17 07:36:36.790728 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:36.790600 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Sep 17 07:36:36.790728 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:36.790550 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 17 07:36:36.790728 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:36.790545 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.24:8816: connect: connection refused" Sep 17 07:36:39.716274 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:39.716218 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Sep 17 07:36:39.716725 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:39.716361 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:36:39.791637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:39.791607 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:39.791845 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:39.791777 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:39.792062 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:39.792049 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:39.792592 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:39.792567 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:39.792692 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:39.792640 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:39.792744 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:39.792692 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:39.798691 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:39.798669 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:40.066839 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:40.066808 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:36:42.726032 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:42.725986 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Sep 17 07:36:42.726524 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:42.726105 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:36:45.020215 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.020187 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:36:45.083039 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.083008 2566 generic.go:356] "Generic (PLEG): container finished" podID="4dab69bc-e18b-4031-8ea7-823b53775599" containerID="53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40" exitCode=137 Sep 17 07:36:45.083039 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.083032 2566 generic.go:356] "Generic (PLEG): container finished" podID="4dab69bc-e18b-4031-8ea7-823b53775599" containerID="eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1" exitCode=137 Sep 17 07:36:45.083230 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.083073 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" event={"ID":"4dab69bc-e18b-4031-8ea7-823b53775599","Type":"ContainerDied","Data":"53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40"} Sep 17 07:36:45.083230 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.083108 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" event={"ID":"4dab69bc-e18b-4031-8ea7-823b53775599","Type":"ContainerDied","Data":"eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1"} Sep 17 07:36:45.083230 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.083121 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" event={"ID":"4dab69bc-e18b-4031-8ea7-823b53775599","Type":"ContainerDied","Data":"9c197e27935a75a7f8e88b840d8dd2d9f95609690919bab4b393e5f8e43d1d5d"} Sep 17 07:36:45.083230 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.083139 2566 scope.go:117] "RemoveContainer" containerID="6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97" Sep 17 07:36:45.083230 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.083074 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm" Sep 17 07:36:45.093430 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.093411 2566 scope.go:117] "RemoveContainer" containerID="53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40" Sep 17 07:36:45.101307 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.101283 2566 scope.go:117] "RemoveContainer" containerID="1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc" Sep 17 07:36:45.108706 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.108684 2566 scope.go:117] "RemoveContainer" containerID="eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1" Sep 17 07:36:45.116457 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.116439 2566 scope.go:117] "RemoveContainer" containerID="c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae" Sep 17 07:36:45.123761 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.123736 2566 scope.go:117] "RemoveContainer" containerID="4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b" Sep 17 07:36:45.129980 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.129952 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-offline-tls\") pod \"4dab69bc-e18b-4031-8ea7-823b53775599\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " Sep 17 07:36:45.130091 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.129984 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-online-tls\") pod \"4dab69bc-e18b-4031-8ea7-823b53775599\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " Sep 17 07:36:45.130091 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.130047 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5287\" (UniqueName: \"kubernetes.io/projected/4dab69bc-e18b-4031-8ea7-823b53775599-kube-api-access-b5287\") pod \"4dab69bc-e18b-4031-8ea7-823b53775599\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " Sep 17 07:36:45.130191 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.130089 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-registry-tls\") pod \"4dab69bc-e18b-4031-8ea7-823b53775599\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " Sep 17 07:36:45.130191 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.130120 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4dab69bc-e18b-4031-8ea7-823b53775599-feast-data\") pod \"4dab69bc-e18b-4031-8ea7-823b53775599\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " Sep 17 07:36:45.130191 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.130160 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-ui-tls\") pod \"4dab69bc-e18b-4031-8ea7-823b53775599\" (UID: \"4dab69bc-e18b-4031-8ea7-823b53775599\") " Sep 17 07:36:45.130801 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.130757 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dab69bc-e18b-4031-8ea7-823b53775599-feast-data" (OuterVolumeSpecName: "feast-data") pod "4dab69bc-e18b-4031-8ea7-823b53775599" (UID: "4dab69bc-e18b-4031-8ea7-823b53775599"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:36:45.132040 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.131771 2566 scope.go:117] "RemoveContainer" containerID="6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97" Sep 17 07:36:45.132432 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.132397 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-online-tls" (OuterVolumeSpecName: "online-tls") pod "4dab69bc-e18b-4031-8ea7-823b53775599" (UID: "4dab69bc-e18b-4031-8ea7-823b53775599"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:36:45.132519 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:36:45.132461 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97\": container with ID starting with 6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97 not found: ID does not exist" containerID="6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97" Sep 17 07:36:45.132579 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.132508 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97"} err="failed to get container status \"6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97\": rpc error: code = NotFound desc = could not find container \"6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97\": container with ID starting with 6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97 not found: ID does not exist" Sep 17 07:36:45.132579 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.132533 2566 scope.go:117] "RemoveContainer" containerID="53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40" Sep 17 07:36:45.132690 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.132632 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4dab69bc-e18b-4031-8ea7-823b53775599" (UID: "4dab69bc-e18b-4031-8ea7-823b53775599"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:36:45.132690 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.132660 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dab69bc-e18b-4031-8ea7-823b53775599-kube-api-access-b5287" (OuterVolumeSpecName: "kube-api-access-b5287") pod "4dab69bc-e18b-4031-8ea7-823b53775599" (UID: "4dab69bc-e18b-4031-8ea7-823b53775599"). InnerVolumeSpecName "kube-api-access-b5287". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:36:45.132808 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.132706 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "4dab69bc-e18b-4031-8ea7-823b53775599" (UID: "4dab69bc-e18b-4031-8ea7-823b53775599"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:36:45.132808 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.132736 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "4dab69bc-e18b-4031-8ea7-823b53775599" (UID: "4dab69bc-e18b-4031-8ea7-823b53775599"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:36:45.132921 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:36:45.132885 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40\": container with ID starting with 53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40 not found: ID does not exist" containerID="53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40" Sep 17 07:36:45.132921 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.132902 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40"} err="failed to get container status \"53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40\": rpc error: code = NotFound desc = could not find container \"53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40\": container with ID starting with 53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40 not found: ID does not exist" Sep 17 07:36:45.132921 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.132915 2566 scope.go:117] "RemoveContainer" containerID="1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc" Sep 17 07:36:45.133183 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:36:45.133162 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc\": container with ID starting with 1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc not found: ID does not exist" containerID="1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc" Sep 17 07:36:45.133239 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.133190 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc"} err="failed to get container status \"1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc\": rpc error: code = NotFound desc = could not find container \"1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc\": container with ID starting with 1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc not found: ID does not exist" Sep 17 07:36:45.133239 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.133206 2566 scope.go:117] "RemoveContainer" containerID="eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1" Sep 17 07:36:45.133480 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:36:45.133461 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1\": container with ID starting with eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1 not found: ID does not exist" containerID="eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1" Sep 17 07:36:45.133538 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.133485 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1"} err="failed to get container status \"eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1\": rpc error: code = NotFound desc = could not find container \"eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1\": container with ID starting with eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1 not found: ID does not exist" Sep 17 07:36:45.133538 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.133498 2566 scope.go:117] "RemoveContainer" containerID="c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae" Sep 17 07:36:45.133752 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:36:45.133736 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae\": container with ID starting with c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae not found: ID does not exist" containerID="c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae" Sep 17 07:36:45.133856 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.133756 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae"} err="failed to get container status \"c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae\": rpc error: code = NotFound desc = could not find container \"c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae\": container with ID starting with c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae not found: ID does not exist" Sep 17 07:36:45.133856 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.133770 2566 scope.go:117] "RemoveContainer" containerID="4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b" Sep 17 07:36:45.134095 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:36:45.134075 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b\": container with ID starting with 4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b not found: ID does not exist" containerID="4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b" Sep 17 07:36:45.134155 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.134098 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b"} err="failed to get container status \"4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b\": rpc error: code = NotFound desc = could not find container \"4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b\": container with ID starting with 4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b not found: ID does not exist" Sep 17 07:36:45.134155 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.134120 2566 scope.go:117] "RemoveContainer" containerID="6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97" Sep 17 07:36:45.134362 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.134345 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97"} err="failed to get container status \"6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97\": rpc error: code = NotFound desc = could not find container \"6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97\": container with ID starting with 6e78208871a7a3a3e7186d03fc4f52f8e52dae55e2d66eed4c00267dc3999e97 not found: ID does not exist" Sep 17 07:36:45.134362 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.134359 2566 scope.go:117] "RemoveContainer" containerID="53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40" Sep 17 07:36:45.134577 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.134562 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40"} err="failed to get container status \"53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40\": rpc error: code = NotFound desc = could not find container \"53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40\": container with ID starting with 53804f51ab0ab66499d9b7dc43ceacc60d21e625c5e5f8be77e63ff8ff3f6b40 not found: ID does not exist" Sep 17 07:36:45.134577 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.134575 2566 scope.go:117] "RemoveContainer" containerID="1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc" Sep 17 07:36:45.134765 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.134751 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc"} err="failed to get container status \"1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc\": rpc error: code = NotFound desc = could not find container \"1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc\": container with ID starting with 1e0b385b23faa6dacac2c98ed9110f68226b03802f21059e5a592946ca9b6fbc not found: ID does not exist" Sep 17 07:36:45.134765 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.134763 2566 scope.go:117] "RemoveContainer" containerID="eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1" Sep 17 07:36:45.134998 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.134981 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1"} err="failed to get container status \"eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1\": rpc error: code = NotFound desc = could not find container \"eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1\": container with ID starting with eba5a0c9a907680d3074b4f6d76bd938b8f61964d009f36db249cd2bf0f2f4e1 not found: ID does not exist" Sep 17 07:36:45.134998 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.134997 2566 scope.go:117] "RemoveContainer" containerID="c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae" Sep 17 07:36:45.135184 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.135168 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae"} err="failed to get container status \"c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae\": rpc error: code = NotFound desc = could not find container \"c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae\": container with ID starting with c98a64b6ad172e03260115b3a3e5c281639514b9d75194dc53a62dc01817daae not found: ID does not exist" Sep 17 07:36:45.135184 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.135183 2566 scope.go:117] "RemoveContainer" containerID="4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b" Sep 17 07:36:45.135371 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.135354 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b"} err="failed to get container status \"4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b\": rpc error: code = NotFound desc = could not find container \"4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b\": container with ID starting with 4b2e8e5827664138e3e4962fd92316f94fb757b184c08ff20d0bb9ec2bb5980b not found: ID does not exist" Sep 17 07:36:45.231027 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.230990 2566 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4dab69bc-e18b-4031-8ea7-823b53775599-feast-data\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:36:45.231027 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.231022 2566 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-ui-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:36:45.231027 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.231030 2566 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-offline-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:36:45.231255 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.231060 2566 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-online-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:36:45.231255 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.231071 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5287\" (UniqueName: \"kubernetes.io/projected/4dab69bc-e18b-4031-8ea7-823b53775599-kube-api-access-b5287\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:36:45.231255 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.231080 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4dab69bc-e18b-4031-8ea7-823b53775599-registry-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:36:45.405314 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.405279 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm"] Sep 17 07:36:45.408251 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:45.408216 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8wpfm"] Sep 17 07:36:46.022475 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:36:46.022440 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" path="/var/lib/kubelet/pods/4dab69bc-e18b-4031-8ea7-823b53775599/volumes" Sep 17 07:37:40.927794 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:40.927738 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:41.029493 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.029451 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:41.287552 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.287465 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:41.388866 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.388828 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:41.399054 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399021 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7"] Sep 17 07:37:41.399281 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399269 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="offline" Sep 17 07:37:41.399331 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399284 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="offline" Sep 17 07:37:41.399331 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399296 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="ui" Sep 17 07:37:41.399331 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399302 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="ui" Sep 17 07:37:41.399331 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399311 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="online" Sep 17 07:37:41.399331 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399316 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="online" Sep 17 07:37:41.399331 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399329 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="feast-init" Sep 17 07:37:41.399501 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399334 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="feast-init" Sep 17 07:37:41.399501 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399341 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="registry" Sep 17 07:37:41.399501 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399360 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="registry" Sep 17 07:37:41.399501 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399369 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="feast-apply" Sep 17 07:37:41.399501 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399374 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="feast-apply" Sep 17 07:37:41.399501 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399416 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="offline" Sep 17 07:37:41.399501 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399423 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="online" Sep 17 07:37:41.399501 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399429 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="registry" Sep 17 07:37:41.399501 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.399436 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dab69bc-e18b-4031-8ea7-823b53775599" containerName="ui" Sep 17 07:37:41.402453 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.402435 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.404871 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.404845 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 17 07:37:41.404998 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.404921 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 17 07:37:41.405064 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.404993 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 17 07:37:41.405826 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.405544 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-9bks2\"" Sep 17 07:37:41.405826 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.405722 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 17 07:37:41.405989 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.405826 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 17 07:37:41.406046 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.405987 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 17 07:37:41.415438 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.415412 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7"] Sep 17 07:37:41.545738 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.545705 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.545950 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.545755 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.545950 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.545776 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r8hf\" (UniqueName: \"kubernetes.io/projected/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-kube-api-access-9r8hf\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.545950 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.545864 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.545950 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.545926 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.546128 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.545953 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.627023 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.626989 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:41.646867 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.646823 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.647029 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.646876 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r8hf\" (UniqueName: \"kubernetes.io/projected/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-kube-api-access-9r8hf\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.647029 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.646904 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.647029 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.646942 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.647029 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.646966 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.647029 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.647012 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.647320 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:37:41.647065 2566 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 17 07:37:41.647320 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:37:41.647105 2566 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 17 07:37:41.647320 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:37:41.647158 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-online-tls podName:9841a7e3-e7b0-4c6b-bd7d-9909f2b35029 nodeName:}" failed. No retries permitted until 2026-09-17 07:37:42.147135146 +0000 UTC m=+1774.639209932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-online-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" (UID: "9841a7e3-e7b0-4c6b-bd7d-9909f2b35029") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 17 07:37:41.647320 ip-10-0-142-90 kubenswrapper[2566]: E0917 07:37:41.647179 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-ui-tls podName:9841a7e3-e7b0-4c6b-bd7d-9909f2b35029 nodeName:}" failed. No retries permitted until 2026-09-17 07:37:42.147169578 +0000 UTC m=+1774.639244352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-ui-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" (UID: "9841a7e3-e7b0-4c6b-bd7d-9909f2b35029") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 17 07:37:41.647521 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.647385 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.647573 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.647539 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.649372 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.649350 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.656173 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.656152 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r8hf\" (UniqueName: \"kubernetes.io/projected/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-kube-api-access-9r8hf\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:41.759069 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.759035 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:41.859992 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:41.859894 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:42.151518 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:42.151434 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:42.151518 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:42.151495 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:42.153866 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:42.153839 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:42.153986 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:42.153950 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-ls9c7\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:42.314314 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:42.314272 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:37:42.590075 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:42.590044 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7"] Sep 17 07:37:42.592089 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:37:42.592060 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9841a7e3_e7b0_4c6b_bd7d_9909f2b35029.slice/crio-16ec3693e505ba4019069751e99912ed776e8e59fac3a79a8296f0526adfdd6d WatchSource:0}: Error finding container 16ec3693e505ba4019069751e99912ed776e8e59fac3a79a8296f0526adfdd6d: Status 404 returned error can't find the container with id 16ec3693e505ba4019069751e99912ed776e8e59fac3a79a8296f0526adfdd6d Sep 17 07:37:43.292970 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:43.292932 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" event={"ID":"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029","Type":"ContainerStarted","Data":"2d997870c0c864fbba46807bc13b7694c9609ccf3cc7811ccf4d5973a7f46f7a"} Sep 17 07:37:43.293367 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:43.292985 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" event={"ID":"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029","Type":"ContainerStarted","Data":"16ec3693e505ba4019069751e99912ed776e8e59fac3a79a8296f0526adfdd6d"} Sep 17 07:37:48.313105 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:48.313070 2566 generic.go:356] "Generic (PLEG): container finished" podID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerID="2d997870c0c864fbba46807bc13b7694c9609ccf3cc7811ccf4d5973a7f46f7a" exitCode=0 Sep 17 07:37:48.313488 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:48.313120 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" event={"ID":"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029","Type":"ContainerDied","Data":"2d997870c0c864fbba46807bc13b7694c9609ccf3cc7811ccf4d5973a7f46f7a"} Sep 17 07:37:49.318210 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:37:49.318176 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" event={"ID":"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029","Type":"ContainerStarted","Data":"2bcd5342d9810a907c70dcc655adad3d73d895d0e571f090708cb3b9e09c59fc"} Sep 17 07:38:08.045685 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:08.045655 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/1.log" Sep 17 07:38:08.046221 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:08.046200 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/1.log" Sep 17 07:38:08.052743 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:08.052718 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:38:08.052971 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:08.052950 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:38:12.407234 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:12.407198 2566 generic.go:356] "Generic (PLEG): container finished" podID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerID="2bcd5342d9810a907c70dcc655adad3d73d895d0e571f090708cb3b9e09c59fc" exitCode=0 Sep 17 07:38:12.407234 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:12.407229 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" event={"ID":"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029","Type":"ContainerDied","Data":"2bcd5342d9810a907c70dcc655adad3d73d895d0e571f090708cb3b9e09c59fc"} Sep 17 07:38:13.415764 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:13.415725 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" event={"ID":"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029","Type":"ContainerStarted","Data":"b96be040afbdb9ad33ab9a82d280838d936d9279ffc4ba01b8b403fd63378baf"} Sep 17 07:38:13.415764 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:13.415777 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" event={"ID":"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029","Type":"ContainerStarted","Data":"8a31d7e4119f5b9740b0c3969cf83f302c0535c3c32a8960a0d6002beeed52b1"} Sep 17 07:38:13.416432 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:13.415806 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" event={"ID":"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029","Type":"ContainerStarted","Data":"6a9eb8d27031600de90fb642ae50bf0c7a6aadb84aed133f20cead05ec327f1d"} Sep 17 07:38:13.439211 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:13.439159 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podStartSLOduration=32.439139787 podStartE2EDuration="32.439139787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:37:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:38:13.438016935 +0000 UTC m=+1805.930091785" watchObservedRunningTime="2026-09-17 07:38:13.439139787 +0000 UTC m=+1805.931214774" Sep 17 07:38:15.315588 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:15.315540 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:38:15.316116 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:15.315618 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:38:15.316116 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:15.315634 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:38:15.317545 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:15.317494 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Sep 17 07:38:15.317708 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:15.317494 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.25:8816: connect: connection refused" Sep 17 07:38:15.317708 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:15.317553 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Sep 17 07:38:18.315297 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:18.315246 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Sep 17 07:38:18.315911 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:18.315247 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Sep 17 07:38:18.315990 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:18.315904 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.25:8816: connect: connection refused" Sep 17 07:38:21.315559 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:21.315525 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:38:21.315947 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:21.315662 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:38:21.316217 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:21.316194 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:38:21.316358 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:21.316289 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:38:21.316442 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:21.316425 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:38:21.316582 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:21.316564 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:38:22.315106 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:22.315047 2566 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Sep 17 07:38:22.315307 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:22.315117 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:38:25.199479 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:25.199448 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:38:26.096281 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:38:26.096242 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:39:27.594755 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:27.594720 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7"] Sep 17 07:39:27.597477 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:27.594968 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="online" containerID="cri-o://6a9eb8d27031600de90fb642ae50bf0c7a6aadb84aed133f20cead05ec327f1d" gracePeriod=30 Sep 17 07:39:27.597477 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:27.595028 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="offline" containerID="cri-o://8a31d7e4119f5b9740b0c3969cf83f302c0535c3c32a8960a0d6002beeed52b1" gracePeriod=30 Sep 17 07:39:27.597477 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:27.595054 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="ui" containerID="cri-o://b96be040afbdb9ad33ab9a82d280838d936d9279ffc4ba01b8b403fd63378baf" gracePeriod=30 Sep 17 07:39:27.625029 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:27.624991 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:39:27.714583 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:27.714545 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:39:27.743482 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:27.743444 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5"] Sep 17 07:39:27.743871 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:27.743755 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="offline" containerID="cri-o://a2b5ae0dd23d3ad8631acfb3024cc2a14c2501105f376e810bab15b17b7d9a5c" gracePeriod=30 Sep 17 07:39:27.743991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:27.743837 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="ui" containerID="cri-o://e4089ea2afd1d79608e5dcb718267fd4048f53a18801696aa1dd673d2d2511e9" gracePeriod=30 Sep 17 07:39:27.744056 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:27.743797 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="online" containerID="cri-o://6c16a72a5625ed21f499d3de9e3b3a271f2e1d2394a08fc364606b62e3f2d7be" gracePeriod=30 Sep 17 07:39:27.744135 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:27.744099 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="registry" containerID="cri-o://465fe87fd7e9691059a1924720554ffd9af689356668a47ca4c8c7a73eb9ff20" gracePeriod=30 Sep 17 07:39:27.786588 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:27.786542 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:39:27.842301 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:27.842253 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:39:28.715464 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:28.715432 2566 generic.go:356] "Generic (PLEG): container finished" podID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerID="e4089ea2afd1d79608e5dcb718267fd4048f53a18801696aa1dd673d2d2511e9" exitCode=0 Sep 17 07:39:28.715897 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:28.715501 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" event={"ID":"87909099-ca75-46bf-93b3-de2c9ca4e102","Type":"ContainerDied","Data":"e4089ea2afd1d79608e5dcb718267fd4048f53a18801696aa1dd673d2d2511e9"} Sep 17 07:39:28.718204 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:28.718177 2566 generic.go:356] "Generic (PLEG): container finished" podID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerID="b96be040afbdb9ad33ab9a82d280838d936d9279ffc4ba01b8b403fd63378baf" exitCode=0 Sep 17 07:39:28.718204 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:28.718196 2566 generic.go:356] "Generic (PLEG): container finished" podID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerID="6a9eb8d27031600de90fb642ae50bf0c7a6aadb84aed133f20cead05ec327f1d" exitCode=0 Sep 17 07:39:28.718399 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:28.718245 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" event={"ID":"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029","Type":"ContainerDied","Data":"b96be040afbdb9ad33ab9a82d280838d936d9279ffc4ba01b8b403fd63378baf"} Sep 17 07:39:28.718399 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:28.718276 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" event={"ID":"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029","Type":"ContainerDied","Data":"6a9eb8d27031600de90fb642ae50bf0c7a6aadb84aed133f20cead05ec327f1d"} Sep 17 07:39:29.725404 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:29.725371 2566 generic.go:356] "Generic (PLEG): container finished" podID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerID="6c16a72a5625ed21f499d3de9e3b3a271f2e1d2394a08fc364606b62e3f2d7be" exitCode=0 Sep 17 07:39:29.725803 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:29.725431 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" event={"ID":"87909099-ca75-46bf-93b3-de2c9ca4e102","Type":"ContainerDied","Data":"6c16a72a5625ed21f499d3de9e3b3a271f2e1d2394a08fc364606b62e3f2d7be"} Sep 17 07:39:29.793206 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:29.793159 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 17 07:39:30.064284 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:30.064243 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Sep 17 07:39:31.316615 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:31.316572 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Sep 17 07:39:32.960476 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:32.960444 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:39:35.200767 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:35.200723 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Sep 17 07:39:39.792509 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:39.792463 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 17 07:39:40.064694 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:40.064602 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Sep 17 07:39:41.317068 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:41.317023 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Sep 17 07:39:45.200911 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:45.200866 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Sep 17 07:39:49.792653 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:49.792599 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Sep 17 07:39:49.793146 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:49.792762 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:39:50.064381 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:50.064292 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Sep 17 07:39:50.064545 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:50.064421 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:39:51.316719 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:51.316670 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.25:8443: connect: connection refused" Sep 17 07:39:51.317159 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:51.316831 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:39:55.201055 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:55.201015 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="online" probeResult="failure" output="Get \"https://10.133.0.25:6567/health\": dial tcp 10.133.0.25:6567: connect: connection refused" Sep 17 07:39:55.201501 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:55.201123 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:39:57.829511 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:57.829477 2566 generic.go:356] "Generic (PLEG): container finished" podID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerID="a2b5ae0dd23d3ad8631acfb3024cc2a14c2501105f376e810bab15b17b7d9a5c" exitCode=137 Sep 17 07:39:57.829511 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:57.829506 2566 generic.go:356] "Generic (PLEG): container finished" podID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerID="465fe87fd7e9691059a1924720554ffd9af689356668a47ca4c8c7a73eb9ff20" exitCode=137 Sep 17 07:39:57.830004 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:57.829546 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" event={"ID":"87909099-ca75-46bf-93b3-de2c9ca4e102","Type":"ContainerDied","Data":"a2b5ae0dd23d3ad8631acfb3024cc2a14c2501105f376e810bab15b17b7d9a5c"} Sep 17 07:39:57.830004 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:57.829589 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" event={"ID":"87909099-ca75-46bf-93b3-de2c9ca4e102","Type":"ContainerDied","Data":"465fe87fd7e9691059a1924720554ffd9af689356668a47ca4c8c7a73eb9ff20"} Sep 17 07:39:57.832172 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:57.832148 2566 generic.go:356] "Generic (PLEG): container finished" podID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerID="8a31d7e4119f5b9740b0c3969cf83f302c0535c3c32a8960a0d6002beeed52b1" exitCode=137 Sep 17 07:39:57.832277 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:57.832217 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" event={"ID":"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029","Type":"ContainerDied","Data":"8a31d7e4119f5b9740b0c3969cf83f302c0535c3c32a8960a0d6002beeed52b1"} Sep 17 07:39:58.246829 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.246768 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:39:58.379779 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.379753 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:39:58.384315 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.384295 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-online-tls\") pod \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " Sep 17 07:39:58.384452 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.384361 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-offline-tls\") pod \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " Sep 17 07:39:58.384452 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.384395 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-feast-data\") pod \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " Sep 17 07:39:58.384452 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.384419 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-registry-tls\") pod \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " Sep 17 07:39:58.384452 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.384444 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9r8hf\" (UniqueName: \"kubernetes.io/projected/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-kube-api-access-9r8hf\") pod \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " Sep 17 07:39:58.384654 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.384487 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-ui-tls\") pod \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\" (UID: \"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029\") " Sep 17 07:39:58.384828 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.384802 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" (UID: "9841a7e3-e7b0-4c6b-bd7d-9909f2b35029"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:39:58.385024 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.385003 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-feast-data" (OuterVolumeSpecName: "feast-data") pod "9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" (UID: "9841a7e3-e7b0-4c6b-bd7d-9909f2b35029"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:39:58.386613 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.386583 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" (UID: "9841a7e3-e7b0-4c6b-bd7d-9909f2b35029"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:39:58.386707 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.386635 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-kube-api-access-9r8hf" (OuterVolumeSpecName: "kube-api-access-9r8hf") pod "9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" (UID: "9841a7e3-e7b0-4c6b-bd7d-9909f2b35029"). InnerVolumeSpecName "kube-api-access-9r8hf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:39:58.386845 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.386823 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-online-tls" (OuterVolumeSpecName: "online-tls") pod "9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" (UID: "9841a7e3-e7b0-4c6b-bd7d-9909f2b35029"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:39:58.386953 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.386937 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" (UID: "9841a7e3-e7b0-4c6b-bd7d-9909f2b35029"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:39:58.485508 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.485471 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-online-tls\") pod \"87909099-ca75-46bf-93b3-de2c9ca4e102\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " Sep 17 07:39:58.485508 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.485517 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-registry-tls\") pod \"87909099-ca75-46bf-93b3-de2c9ca4e102\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " Sep 17 07:39:58.485794 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.485541 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlmjj\" (UniqueName: \"kubernetes.io/projected/87909099-ca75-46bf-93b3-de2c9ca4e102-kube-api-access-hlmjj\") pod \"87909099-ca75-46bf-93b3-de2c9ca4e102\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " Sep 17 07:39:58.485794 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.485574 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-offline-tls\") pod \"87909099-ca75-46bf-93b3-de2c9ca4e102\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " Sep 17 07:39:58.485794 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.485592 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/87909099-ca75-46bf-93b3-de2c9ca4e102-feast-data\") pod \"87909099-ca75-46bf-93b3-de2c9ca4e102\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " Sep 17 07:39:58.485794 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.485642 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-ui-tls\") pod \"87909099-ca75-46bf-93b3-de2c9ca4e102\" (UID: \"87909099-ca75-46bf-93b3-de2c9ca4e102\") " Sep 17 07:39:58.486014 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.485839 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-registry-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:39:58.486014 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.485860 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r8hf\" (UniqueName: \"kubernetes.io/projected/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-kube-api-access-9r8hf\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:39:58.486014 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.485874 2566 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-ui-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:39:58.486014 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.485887 2566 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-online-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:39:58.486014 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.485900 2566 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-offline-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:39:58.486014 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.485912 2566 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029-feast-data\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:39:58.486332 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.486304 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87909099-ca75-46bf-93b3-de2c9ca4e102-feast-data" (OuterVolumeSpecName: "feast-data") pod "87909099-ca75-46bf-93b3-de2c9ca4e102" (UID: "87909099-ca75-46bf-93b3-de2c9ca4e102"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:39:58.488032 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.488003 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "87909099-ca75-46bf-93b3-de2c9ca4e102" (UID: "87909099-ca75-46bf-93b3-de2c9ca4e102"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:39:58.488321 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.488300 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87909099-ca75-46bf-93b3-de2c9ca4e102-kube-api-access-hlmjj" (OuterVolumeSpecName: "kube-api-access-hlmjj") pod "87909099-ca75-46bf-93b3-de2c9ca4e102" (UID: "87909099-ca75-46bf-93b3-de2c9ca4e102"). InnerVolumeSpecName "kube-api-access-hlmjj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:39:58.488321 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.488302 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "87909099-ca75-46bf-93b3-de2c9ca4e102" (UID: "87909099-ca75-46bf-93b3-de2c9ca4e102"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:39:58.488442 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.488319 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "87909099-ca75-46bf-93b3-de2c9ca4e102" (UID: "87909099-ca75-46bf-93b3-de2c9ca4e102"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:39:58.488442 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.488331 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-online-tls" (OuterVolumeSpecName: "online-tls") pod "87909099-ca75-46bf-93b3-de2c9ca4e102" (UID: "87909099-ca75-46bf-93b3-de2c9ca4e102"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:39:58.586764 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.586719 2566 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-ui-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:39:58.586764 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.586756 2566 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-online-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:39:58.586764 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.586770 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-registry-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:39:58.587106 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.586805 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlmjj\" (UniqueName: \"kubernetes.io/projected/87909099-ca75-46bf-93b3-de2c9ca4e102-kube-api-access-hlmjj\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:39:58.587106 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.586817 2566 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/87909099-ca75-46bf-93b3-de2c9ca4e102-offline-tls\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:39:58.587106 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.586828 2566 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/87909099-ca75-46bf-93b3-de2c9ca4e102-feast-data\") on node \"ip-10-0-142-90.ec2.internal\" DevicePath \"\"" Sep 17 07:39:58.838479 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.838438 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" event={"ID":"87909099-ca75-46bf-93b3-de2c9ca4e102","Type":"ContainerDied","Data":"3b274c04d2f4c964d624c17aa5d9bfd4ec1d3fc967ded9d4ed0ccba314e93b24"} Sep 17 07:39:58.838479 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.838465 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5" Sep 17 07:39:58.839027 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.838507 2566 scope.go:117] "RemoveContainer" containerID="e4089ea2afd1d79608e5dcb718267fd4048f53a18801696aa1dd673d2d2511e9" Sep 17 07:39:58.841296 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.841274 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" event={"ID":"9841a7e3-e7b0-4c6b-bd7d-9909f2b35029","Type":"ContainerDied","Data":"16ec3693e505ba4019069751e99912ed776e8e59fac3a79a8296f0526adfdd6d"} Sep 17 07:39:58.841410 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.841278 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7" Sep 17 07:39:58.847013 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.846991 2566 scope.go:117] "RemoveContainer" containerID="a2b5ae0dd23d3ad8631acfb3024cc2a14c2501105f376e810bab15b17b7d9a5c" Sep 17 07:39:58.854991 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.854971 2566 scope.go:117] "RemoveContainer" containerID="6c16a72a5625ed21f499d3de9e3b3a271f2e1d2394a08fc364606b62e3f2d7be" Sep 17 07:39:58.862642 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.862615 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5"] Sep 17 07:39:58.862842 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.862763 2566 scope.go:117] "RemoveContainer" containerID="465fe87fd7e9691059a1924720554ffd9af689356668a47ca4c8c7a73eb9ff20" Sep 17 07:39:58.868477 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.868451 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9j2m5"] Sep 17 07:39:58.872262 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.872231 2566 scope.go:117] "RemoveContainer" containerID="69fa8d411db6433f80fc39835e4d6cdfc635380ac09dbf84efd52b7956dea70d" Sep 17 07:39:58.879119 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.879103 2566 scope.go:117] "RemoveContainer" containerID="1a3a7e08960a413afa52c26cc51a367d0b0c1cc826893cd2c07129eb807148e1" Sep 17 07:39:58.881672 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.881652 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7"] Sep 17 07:39:58.886100 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.886072 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-ls9c7"] Sep 17 07:39:58.886772 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.886760 2566 scope.go:117] "RemoveContainer" containerID="b96be040afbdb9ad33ab9a82d280838d936d9279ffc4ba01b8b403fd63378baf" Sep 17 07:39:58.893537 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.893519 2566 scope.go:117] "RemoveContainer" containerID="8a31d7e4119f5b9740b0c3969cf83f302c0535c3c32a8960a0d6002beeed52b1" Sep 17 07:39:58.901077 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.901052 2566 scope.go:117] "RemoveContainer" containerID="6a9eb8d27031600de90fb642ae50bf0c7a6aadb84aed133f20cead05ec327f1d" Sep 17 07:39:58.908258 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.908235 2566 scope.go:117] "RemoveContainer" containerID="2bcd5342d9810a907c70dcc655adad3d73d895d0e571f090708cb3b9e09c59fc" Sep 17 07:39:58.915068 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:39:58.915050 2566 scope.go:117] "RemoveContainer" containerID="2d997870c0c864fbba46807bc13b7694c9609ccf3cc7811ccf4d5973a7f46f7a" Sep 17 07:40:00.022388 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:00.022355 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" path="/var/lib/kubelet/pods/87909099-ca75-46bf-93b3-de2c9ca4e102/volumes" Sep 17 07:40:00.022966 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:00.022952 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" path="/var/lib/kubelet/pods/9841a7e3-e7b0-4c6b-bd7d-9909f2b35029/volumes" Sep 17 07:40:00.536776 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:00.536738 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:40:00.661970 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:00.661922 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:40:06.448522 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:06.448487 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:40:06.550264 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:06.550232 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:40:06.945601 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:06.945567 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:40:07.099431 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:07.099395 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:40:19.790974 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:19.790941 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:40:19.892100 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:19.892072 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:40:25.321210 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.321174 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:40:25.453979 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.453943 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vh9zj/must-gather-drlh4"] Sep 17 07:40:25.454231 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454216 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="offline" Sep 17 07:40:25.454231 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454233 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="offline" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454247 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="feast-init" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454255 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="feast-init" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454266 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="registry" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454275 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="registry" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454285 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="ui" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454291 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="ui" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454297 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="feast-apply" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454302 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="feast-apply" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454309 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="offline" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454314 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="offline" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454319 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="ui" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454323 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="ui" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454331 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="online" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454336 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="online" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454345 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="online" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454350 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="online" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454356 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="feast-init" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454360 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="feast-init" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454367 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="feast-apply" Sep 17 07:40:25.454380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454375 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="feast-apply" Sep 17 07:40:25.454994 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454416 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="offline" Sep 17 07:40:25.454994 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454424 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="ui" Sep 17 07:40:25.454994 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454430 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="registry" Sep 17 07:40:25.454994 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454438 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="offline" Sep 17 07:40:25.454994 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454448 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="online" Sep 17 07:40:25.454994 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454456 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9841a7e3-e7b0-4c6b-bd7d-9909f2b35029" containerName="ui" Sep 17 07:40:25.454994 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.454462 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="87909099-ca75-46bf-93b3-de2c9ca4e102" containerName="online" Sep 17 07:40:25.457232 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.457216 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vh9zj/must-gather-drlh4" Sep 17 07:40:25.459504 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.459484 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vh9zj\"/\"default-dockercfg-79wf9\"" Sep 17 07:40:25.459621 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.459573 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vh9zj\"/\"openshift-service-ca.crt\"" Sep 17 07:40:25.460144 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.460129 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vh9zj\"/\"kube-root-ca.crt\"" Sep 17 07:40:25.466148 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.466126 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vh9zj/must-gather-drlh4"] Sep 17 07:40:25.468516 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.468494 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:40:25.541449 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.541411 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:40:25.581207 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.581121 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8e779596-e742-48e7-8777-5178c034dadb-must-gather-output\") pod \"must-gather-drlh4\" (UID: \"8e779596-e742-48e7-8777-5178c034dadb\") " pod="openshift-must-gather-vh9zj/must-gather-drlh4" Sep 17 07:40:25.581207 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.581159 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6smd\" (UniqueName: \"kubernetes.io/projected/8e779596-e742-48e7-8777-5178c034dadb-kube-api-access-b6smd\") pod \"must-gather-drlh4\" (UID: \"8e779596-e742-48e7-8777-5178c034dadb\") " pod="openshift-must-gather-vh9zj/must-gather-drlh4" Sep 17 07:40:25.682325 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.682278 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8e779596-e742-48e7-8777-5178c034dadb-must-gather-output\") pod \"must-gather-drlh4\" (UID: \"8e779596-e742-48e7-8777-5178c034dadb\") " pod="openshift-must-gather-vh9zj/must-gather-drlh4" Sep 17 07:40:25.682325 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.682327 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6smd\" (UniqueName: \"kubernetes.io/projected/8e779596-e742-48e7-8777-5178c034dadb-kube-api-access-b6smd\") pod \"must-gather-drlh4\" (UID: \"8e779596-e742-48e7-8777-5178c034dadb\") " pod="openshift-must-gather-vh9zj/must-gather-drlh4" Sep 17 07:40:25.682661 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.682638 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8e779596-e742-48e7-8777-5178c034dadb-must-gather-output\") pod \"must-gather-drlh4\" (UID: \"8e779596-e742-48e7-8777-5178c034dadb\") " pod="openshift-must-gather-vh9zj/must-gather-drlh4" Sep 17 07:40:25.691488 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.691451 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6smd\" (UniqueName: \"kubernetes.io/projected/8e779596-e742-48e7-8777-5178c034dadb-kube-api-access-b6smd\") pod \"must-gather-drlh4\" (UID: \"8e779596-e742-48e7-8777-5178c034dadb\") " pod="openshift-must-gather-vh9zj/must-gather-drlh4" Sep 17 07:40:25.765372 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:25.765326 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vh9zj/must-gather-drlh4" Sep 17 07:40:26.039901 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:26.039779 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vh9zj/must-gather-drlh4"] Sep 17 07:40:26.042658 ip-10-0-142-90 kubenswrapper[2566]: W0917 07:40:26.042628 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e779596_e742_48e7_8777_5178c034dadb.slice/crio-66607fe3dd0d9355cb76bf806dd111ec816272bbc559fd840a330f9691f64d2c WatchSource:0}: Error finding container 66607fe3dd0d9355cb76bf806dd111ec816272bbc559fd840a330f9691f64d2c: Status 404 returned error can't find the container with id 66607fe3dd0d9355cb76bf806dd111ec816272bbc559fd840a330f9691f64d2c Sep 17 07:40:26.044637 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:26.044621 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:40:26.940954 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:26.940908 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vh9zj/must-gather-drlh4" event={"ID":"8e779596-e742-48e7-8777-5178c034dadb","Type":"ContainerStarted","Data":"4b4fe812908e6a5e6efc15957fe67155bc444dcac5581a2f3e68571c6dbcda71"} Sep 17 07:40:26.941367 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:26.940973 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vh9zj/must-gather-drlh4" event={"ID":"8e779596-e742-48e7-8777-5178c034dadb","Type":"ContainerStarted","Data":"66607fe3dd0d9355cb76bf806dd111ec816272bbc559fd840a330f9691f64d2c"} Sep 17 07:40:27.949251 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:27.949216 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vh9zj/must-gather-drlh4" event={"ID":"8e779596-e742-48e7-8777-5178c034dadb","Type":"ContainerStarted","Data":"5fed1274fb08767136fb4fa9e15c5781e3851d1ea333d42a728a5df0cc636b17"} Sep 17 07:40:27.966613 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:27.966561 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vh9zj/must-gather-drlh4" podStartSLOduration=2.231617214 podStartE2EDuration="2.966540545s" totalImagesPullingTime="734.923331ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:40:27.965154807 +0000 UTC m=+1940.457229601" watchObservedRunningTime="2026-09-17 07:40:27.966540545 +0000 UTC m=+1940.458615337" Sep 17 07:40:28.222476 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:28.222375 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7jt2z_e59c89a5-bae3-48e6-bdb5-f52094590f45/global-pull-secret-syncer/0.log" Sep 17 07:40:28.378390 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:28.378358 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" Sep 17 07:40:28.520897 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:28.520808 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-90.ec2.internal_b5d2ed14de66e6a088f1d600ae6da75a/haproxy/0.log" Sep 17 07:40:30.874580 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:30.874531 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:40:30.979692 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:30.979660 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mr6f_a9bab8de-9d11-4e20-814c-74e2afb9eb19/konnectivity-agent/0.log" limit=1024 Sep 17 07:40:32.495928 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:32.495896 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6fk95_0778a52f-26ea-49d3-b3f6-8ecf5e874e85/node-exporter/0.log" Sep 17 07:40:32.520480 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:32.520451 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6fk95_0778a52f-26ea-49d3-b3f6-8ecf5e874e85/kube-rbac-proxy/0.log" Sep 17 07:40:32.554205 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:32.554178 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6fk95_0778a52f-26ea-49d3-b3f6-8ecf5e874e85/init-textfile/0.log" Sep 17 07:40:35.001706 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.001674 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/1.log" Sep 17 07:40:35.009565 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.009536 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-9n87p_af922fc6-a4ff-40a5-a5d7-d86698d0a1fb/console-operator/2.log" Sep 17 07:40:35.594381 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.594350 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq"] Sep 17 07:40:35.599589 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.599561 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:35.611316 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.611284 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq"] Sep 17 07:40:35.766188 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.766154 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3ad1d6b-53fd-444d-9755-082b3ddc20ac-lib-modules\") pod \"perf-node-gather-daemonset-44vjq\" (UID: \"c3ad1d6b-53fd-444d-9755-082b3ddc20ac\") " pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:35.766351 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.766200 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c3ad1d6b-53fd-444d-9755-082b3ddc20ac-proc\") pod \"perf-node-gather-daemonset-44vjq\" (UID: \"c3ad1d6b-53fd-444d-9755-082b3ddc20ac\") " pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:35.766351 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.766278 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3ad1d6b-53fd-444d-9755-082b3ddc20ac-sys\") pod \"perf-node-gather-daemonset-44vjq\" (UID: \"c3ad1d6b-53fd-444d-9755-082b3ddc20ac\") " pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:35.766351 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.766334 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c3ad1d6b-53fd-444d-9755-082b3ddc20ac-podres\") pod \"perf-node-gather-daemonset-44vjq\" (UID: \"c3ad1d6b-53fd-444d-9755-082b3ddc20ac\") " pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:35.766475 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.766379 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr2p9\" (UniqueName: \"kubernetes.io/projected/c3ad1d6b-53fd-444d-9755-082b3ddc20ac-kube-api-access-gr2p9\") pod \"perf-node-gather-daemonset-44vjq\" (UID: \"c3ad1d6b-53fd-444d-9755-082b3ddc20ac\") " pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:35.867673 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.867579 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3ad1d6b-53fd-444d-9755-082b3ddc20ac-sys\") pod \"perf-node-gather-daemonset-44vjq\" (UID: \"c3ad1d6b-53fd-444d-9755-082b3ddc20ac\") " pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:35.867673 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.867626 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c3ad1d6b-53fd-444d-9755-082b3ddc20ac-podres\") pod \"perf-node-gather-daemonset-44vjq\" (UID: \"c3ad1d6b-53fd-444d-9755-082b3ddc20ac\") " pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:35.867938 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.867679 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr2p9\" (UniqueName: \"kubernetes.io/projected/c3ad1d6b-53fd-444d-9755-082b3ddc20ac-kube-api-access-gr2p9\") pod \"perf-node-gather-daemonset-44vjq\" (UID: \"c3ad1d6b-53fd-444d-9755-082b3ddc20ac\") " pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:35.867938 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.867714 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3ad1d6b-53fd-444d-9755-082b3ddc20ac-sys\") pod \"perf-node-gather-daemonset-44vjq\" (UID: \"c3ad1d6b-53fd-444d-9755-082b3ddc20ac\") " pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:35.867938 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.867717 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3ad1d6b-53fd-444d-9755-082b3ddc20ac-lib-modules\") pod \"perf-node-gather-daemonset-44vjq\" (UID: \"c3ad1d6b-53fd-444d-9755-082b3ddc20ac\") " pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:35.867938 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.867792 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c3ad1d6b-53fd-444d-9755-082b3ddc20ac-proc\") pod \"perf-node-gather-daemonset-44vjq\" (UID: \"c3ad1d6b-53fd-444d-9755-082b3ddc20ac\") " pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:35.867938 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.867801 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c3ad1d6b-53fd-444d-9755-082b3ddc20ac-podres\") pod \"perf-node-gather-daemonset-44vjq\" (UID: \"c3ad1d6b-53fd-444d-9755-082b3ddc20ac\") " pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:35.867938 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.867834 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3ad1d6b-53fd-444d-9755-082b3ddc20ac-lib-modules\") pod \"perf-node-gather-daemonset-44vjq\" (UID: \"c3ad1d6b-53fd-444d-9755-082b3ddc20ac\") " pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:35.867938 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.867862 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c3ad1d6b-53fd-444d-9755-082b3ddc20ac-proc\") pod \"perf-node-gather-daemonset-44vjq\" (UID: \"c3ad1d6b-53fd-444d-9755-082b3ddc20ac\") " pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:35.876559 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.876499 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr2p9\" (UniqueName: \"kubernetes.io/projected/c3ad1d6b-53fd-444d-9755-082b3ddc20ac-kube-api-access-gr2p9\") pod \"perf-node-gather-daemonset-44vjq\" (UID: \"c3ad1d6b-53fd-444d-9755-082b3ddc20ac\") " pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:35.912236 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:35.912183 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:36.203623 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:36.203524 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq"] Sep 17 07:40:36.820380 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:36.820350 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-79g8v_a647a2aa-ba1d-4809-9639-dddcba088b0e/dns/0.log" Sep 17 07:40:36.840361 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:36.840315 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-79g8v_a647a2aa-ba1d-4809-9639-dddcba088b0e/kube-rbac-proxy/0.log" Sep 17 07:40:36.990182 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:36.990142 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" event={"ID":"c3ad1d6b-53fd-444d-9755-082b3ddc20ac","Type":"ContainerStarted","Data":"08212f12a6ac744fadea070cd20d91c810cfd370c047c1199b018102a1ca3d37"} Sep 17 07:40:36.990357 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:36.990203 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" event={"ID":"c3ad1d6b-53fd-444d-9755-082b3ddc20ac","Type":"ContainerStarted","Data":"427d3f43c0995b26a332baacb43aeaa1737ed6da32c24c267789dbe61108368d"} Sep 17 07:40:36.990357 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:36.990239 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:36.997011 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:36.996990 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wffc2_e94051ac-fe49-4338-ab5f-69979421a9cb/dns-node-resolver/0.log" Sep 17 07:40:37.006085 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:37.006043 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" podStartSLOduration=2.006028648 podStartE2EDuration="2.006028648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:40:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:40:37.005364193 +0000 UTC m=+1949.497438986" watchObservedRunningTime="2026-09-17 07:40:37.006028648 +0000 UTC m=+1949.498103441" Sep 17 07:40:37.443302 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:37.443275 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kvftr_7a9610b5-26e5-44c9-a253-ea18eaec24af/node-ca/0.log" Sep 17 07:40:38.632547 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:38.632520 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2n76n_f70632fa-c15c-4beb-8c27-2b4316baafd5/serve-healthcheck-canary/0.log" Sep 17 07:40:39.250172 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:39.250141 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j8x88_a9256b15-bf3a-4cad-8894-6ca06e0b7fe2/kube-rbac-proxy/0.log" Sep 17 07:40:39.272437 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:39.272408 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j8x88_a9256b15-bf3a-4cad-8894-6ca06e0b7fe2/exporter/0.log" Sep 17 07:40:39.295619 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:39.295586 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j8x88_a9256b15-bf3a-4cad-8894-6ca06e0b7fe2/extractor/0.log" Sep 17 07:40:43.002138 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:43.002106 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vh9zj/perf-node-gather-daemonset-44vjq" Sep 17 07:40:45.084979 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:45.084952 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzr67_8afb189f-1edd-473f-92dc-f923a60d691e/kube-multus-additional-cni-plugins/0.log" Sep 17 07:40:45.112060 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:45.112029 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzr67_8afb189f-1edd-473f-92dc-f923a60d691e/egress-router-binary-copy/0.log" Sep 17 07:40:45.138977 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:45.138950 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzr67_8afb189f-1edd-473f-92dc-f923a60d691e/cni-plugins/0.log" Sep 17 07:40:45.164007 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:45.163971 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzr67_8afb189f-1edd-473f-92dc-f923a60d691e/bond-cni-plugin/0.log" Sep 17 07:40:45.189015 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:45.188983 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzr67_8afb189f-1edd-473f-92dc-f923a60d691e/routeoverride-cni/0.log" Sep 17 07:40:45.215689 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:45.215659 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzr67_8afb189f-1edd-473f-92dc-f923a60d691e/whereabouts-cni-bincopy/0.log" Sep 17 07:40:45.240302 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:45.240272 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzr67_8afb189f-1edd-473f-92dc-f923a60d691e/whereabouts-cni/0.log" Sep 17 07:40:45.491848 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:45.491753 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nx9rp_93339a00-d0f0-4f0d-9449-bbb4566c2b93/kube-multus/0.log" Sep 17 07:40:45.597496 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:45.597458 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-z2l2s_dc4c353d-0b67-4b66-a052-dd8397466d6f/network-metrics-daemon/0.log" Sep 17 07:40:45.616741 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:45.616699 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-z2l2s_dc4c353d-0b67-4b66-a052-dd8397466d6f/kube-rbac-proxy/0.log" Sep 17 07:40:46.675472 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:46.675377 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-controller/0.log" Sep 17 07:40:46.692910 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:46.692879 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/1.log" Sep 17 07:40:46.693059 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:46.693011 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovn-acl-logging/0.log" Sep 17 07:40:46.715181 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:46.715154 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/kube-rbac-proxy-node/0.log" Sep 17 07:40:46.741948 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:46.741921 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/kube-rbac-proxy-ovn-metrics/0.log" Sep 17 07:40:46.765663 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:46.765632 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/northd/0.log" Sep 17 07:40:46.786515 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:46.786487 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/nbdb/0.log" Sep 17 07:40:46.811054 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:46.811021 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/sbdb/0.log" Sep 17 07:40:46.864294 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:46.864259 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64lxj_3953be2c-e8eb-4172-88fa-8d04babc59fd/ovnkube-controller/0.log" Sep 17 07:40:48.044328 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:48.044285 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-6f8b768455-t7hp8_843c9d67-6674-4385-9d84-bdad650d593c/check-endpoints/0.log" Sep 17 07:40:48.115188 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:48.115152 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-vddmm_3b975ace-eff0-4687-845a-c9607f77ac13/network-check-target-container/0.log" Sep 17 07:40:49.004628 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:49.004603 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-8cl5p_abd2409d-642b-4c1e-a55c-a4a2c702b553/iptables-alerter/0.log" Sep 17 07:40:49.706584 ip-10-0-142-90 kubenswrapper[2566]: I0917 07:40:49.706551 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-zslxf_b815c071-1510-4f92-a8be-9d9a49cba65a/tuned/0.log"