Sep 08 10:17:03.520811 ip-10-0-132-203 systemd[1]: Starting Kubernetes Kubelet... Sep 08 10:17:03.940608 ip-10-0-132-203 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 10:17:03.940608 ip-10-0-132-203 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 08 10:17:03.940608 ip-10-0-132-203 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 10:17:03.940608 ip-10-0-132-203 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 08 10:17:03.940608 ip-10-0-132-203 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 10:17:03.942785 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.942692 2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 08 10:17:03.947031 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947011 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 10:17:03.947031 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947029 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 10:17:03.947031 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947034 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 10:17:03.947031 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947039 2561 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947043 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947048 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947052 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947057 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947063 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947068 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947073 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947077 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947138 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947190 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947195 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947200 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947205 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947209 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947214 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947218 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947222 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947227 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 10:17:03.947264 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947234 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 10:17:03.948033 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947241 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 10:17:03.948033 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947246 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 10:17:03.948033 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.947251 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 10:17:03.949323 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949301 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 10:17:03.949441 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949430 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949443 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949449 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949452 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949455 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949458 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949461 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949464 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949467 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949470 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949473 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949475 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949478 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949481 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949483 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949486 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949489 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949492 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949495 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 10:17:03.949489 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949498 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949502 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949505 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949508 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949511 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949513 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949516 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949518 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949521 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949541 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949546 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949550 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949552 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949555 2561 feature_gate.go:328] unrecognized feature gate: Example Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949558 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949563 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949566 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949568 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949571 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949573 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 10:17:03.950028 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949576 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949579 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949582 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949584 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949587 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949590 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949592 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949595 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949597 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949600 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949603 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949605 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949607 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949610 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949614 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949616 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949619 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949621 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949624 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949627 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 10:17:03.950545 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949630 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949632 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949635 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949637 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949640 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949642 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949645 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949647 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949650 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949652 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949655 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949657 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.949660 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950385 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950392 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950395 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950398 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950401 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950404 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950406 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 10:17:03.951057 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950409 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950412 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950414 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950417 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950420 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950423 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950427 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950430 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950433 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950436 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950438 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950441 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950444 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950446 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950449 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950451 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950454 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950457 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950459 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 10:17:03.951595 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950462 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950464 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950467 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950470 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950473 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950475 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950478 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950480 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950483 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950485 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950488 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950490 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950493 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950496 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950498 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950501 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950505 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950509 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950513 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 10:17:03.952121 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950517 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950519 2561 feature_gate.go:328] unrecognized feature gate: Example Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950522 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950540 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950544 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950546 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950549 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950552 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950554 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950557 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950560 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950562 2561 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950565 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950567 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950570 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950573 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950576 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950579 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950582 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950584 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950587 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 10:17:03.952611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950589 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950594 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950598 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950601 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950604 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950607 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950609 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950612 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950615 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950617 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950620 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950623 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950625 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950628 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950630 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950633 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950635 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950638 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950640 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 10:17:03.953132 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950643 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950645 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950648 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950650 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950653 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950657 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950661 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950665 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950667 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950670 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950673 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950676 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950679 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.950681 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951113 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951118 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951122 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951125 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951129 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951131 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 10:17:03.953616 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951134 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951137 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951139 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951142 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951145 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951148 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951151 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951154 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951157 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951160 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951162 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951165 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951167 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951171 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951175 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951177 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951180 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951183 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951185 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 10:17:03.954131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951188 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951191 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951193 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951197 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951200 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951202 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951205 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951207 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951210 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951212 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951215 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951218 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951220 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951222 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951225 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951228 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951231 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951236 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951239 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951241 2561 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951244 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 10:17:03.954611 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951246 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951249 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951251 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951255 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951258 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951260 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951264 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951267 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951271 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951274 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951277 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951279 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951282 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951285 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951287 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951290 2561 feature_gate.go:328] unrecognized feature gate: Example Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951293 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951295 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951297 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 10:17:03.955131 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951300 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951302 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951305 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951307 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951310 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951312 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951315 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951317 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951320 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951323 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951326 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951328 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951331 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951333 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951336 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951339 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951341 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951344 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951346 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951349 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 10:17:03.955620 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951352 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951354 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951359 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951362 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951365 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951368 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951371 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951374 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951377 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951380 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951383 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951385 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951388 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.951391 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951463 2561 flags.go:64] FLAG: --address="0.0.0.0" Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951505 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951512 2561 flags.go:64] FLAG: --anonymous-auth="true" Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951517 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951522 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951539 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 08 10:17:03.956105 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951545 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951553 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951556 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951559 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951563 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951566 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951569 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951572 2561 flags.go:64] FLAG: --cgroup-root="" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951575 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951578 2561 flags.go:64] FLAG: --client-ca-file="" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951581 2561 flags.go:64] FLAG: --cloud-provider="external" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951584 2561 flags.go:64] FLAG: --cluster-dns="[]" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951589 2561 flags.go:64] FLAG: --cluster-domain="" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951592 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951595 2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951599 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951602 2561 flags.go:64] FLAG: --container-log-max-files="5" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951606 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951609 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951612 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951616 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951619 2561 flags.go:64] FLAG: --contention-profiling="false" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951622 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951625 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 08 10:17:03.956694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951629 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951631 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951636 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951639 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951641 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951644 2561 flags.go:64] FLAG: --enable-load-reader="false" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951647 2561 flags.go:64] FLAG: --enable-server="true" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951650 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951655 2561 flags.go:64] FLAG: --event-burst="100" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951658 2561 flags.go:64] FLAG: --event-qps="50" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951661 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951664 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951667 2561 flags.go:64] FLAG: --eviction-hard="" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951671 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951673 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951676 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951679 2561 flags.go:64] FLAG: --eviction-soft="" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951682 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951685 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951687 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951690 2561 flags.go:64] FLAG: --experimental-mounter-path="" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951693 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951696 2561 flags.go:64] FLAG: --fail-swap-on="true" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951699 2561 flags.go:64] FLAG: --feature-gates="" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951703 2561 flags.go:64] FLAG: --file-check-frequency="20s" Sep 08 10:17:03.957306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951705 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951708 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951712 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951715 2561 flags.go:64] FLAG: --healthz-port="10248" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951717 2561 flags.go:64] FLAG: --help="false" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951720 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-132-203.ec2.internal" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951724 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951727 2561 flags.go:64] FLAG: --http-check-frequency="20s" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951729 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951733 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951736 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951739 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951742 2561 flags.go:64] FLAG: --image-service-endpoint="" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951745 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951748 2561 flags.go:64] FLAG: --kube-api-burst="100" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951750 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951754 2561 flags.go:64] FLAG: --kube-api-qps="50" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951757 2561 flags.go:64] FLAG: --kube-reserved="" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951760 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951763 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951766 2561 flags.go:64] FLAG: --kubelet-cgroups="" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951769 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951772 2561 flags.go:64] FLAG: --lock-file="" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951775 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 08 10:17:03.957994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951777 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951780 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951786 2561 flags.go:64] FLAG: --log-json-split-stream="false" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951791 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951794 2561 flags.go:64] FLAG: --log-text-split-stream="false" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951796 2561 flags.go:64] FLAG: --logging-format="text" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951799 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951803 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951806 2561 flags.go:64] FLAG: --manifest-url="" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951809 2561 flags.go:64] FLAG: --manifest-url-header="" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951813 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951816 2561 flags.go:64] FLAG: --max-open-files="1000000" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951820 2561 flags.go:64] FLAG: --max-pods="110" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951823 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951826 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951829 2561 flags.go:64] FLAG: --memory-manager-policy="None" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951832 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951835 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951838 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951841 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951848 2561 flags.go:64] FLAG: --node-status-max-images="50" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951851 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951854 2561 flags.go:64] FLAG: --oom-score-adj="-999" Sep 08 10:17:03.958628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951857 2561 flags.go:64] FLAG: --pod-cidr="" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951860 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951866 2561 flags.go:64] FLAG: --pod-manifest-path="" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951869 2561 flags.go:64] FLAG: --pod-max-pids="-1" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951872 2561 flags.go:64] FLAG: --pods-per-core="0" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951875 2561 flags.go:64] FLAG: --port="10250" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951878 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951881 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cd545c5ff1389c1a" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951884 2561 flags.go:64] FLAG: --qos-reserved="" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951887 2561 flags.go:64] FLAG: --read-only-port="10255" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951890 2561 flags.go:64] FLAG: --register-node="true" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951893 2561 flags.go:64] FLAG: --register-with-taints="" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951898 2561 flags.go:64] FLAG: --registry-burst="10" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951900 2561 flags.go:64] FLAG: --registry-qps="5" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951903 2561 flags.go:64] FLAG: --reserved-cpus="" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951906 2561 flags.go:64] FLAG: --reserved-memory="" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951910 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951913 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951916 2561 flags.go:64] FLAG: --rotate-certificates="false" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951919 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951921 2561 flags.go:64] FLAG: --runonce="false" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951924 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951927 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951930 2561 flags.go:64] FLAG: --seccomp-default="false" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951933 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 08 10:17:03.959198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951936 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951939 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951942 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951945 2561 flags.go:64] FLAG: --storage-driver-password="root" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951948 2561 flags.go:64] FLAG: --storage-driver-secure="false" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951951 2561 flags.go:64] FLAG: --storage-driver-table="stats" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951954 2561 flags.go:64] FLAG: --storage-driver-user="root" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951956 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951959 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951962 2561 flags.go:64] FLAG: --system-cgroups="" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951965 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951970 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951973 2561 flags.go:64] FLAG: --tls-cert-file="" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951976 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951981 2561 flags.go:64] FLAG: --tls-min-version="" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951984 2561 flags.go:64] FLAG: --tls-private-key-file="" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951986 2561 flags.go:64] FLAG: --topology-manager-policy="none" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951989 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951992 2561 flags.go:64] FLAG: --topology-manager-scope="container" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.951997 2561 flags.go:64] FLAG: --v="2" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.952001 2561 flags.go:64] FLAG: --version="false" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.952004 2561 flags.go:64] FLAG: --vmodule="" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.952008 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.952011 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 08 10:17:03.959839 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952129 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952132 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952136 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952139 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952141 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952144 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952149 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952152 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952154 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952158 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952160 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952163 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952166 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952169 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952171 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952174 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952176 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952179 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952182 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952184 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 10:17:03.960437 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952188 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952191 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952194 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952197 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952199 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952202 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952205 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952209 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952212 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952214 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952217 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952219 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952222 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952225 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952227 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952230 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952232 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952235 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952239 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952242 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 10:17:03.960984 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952244 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952247 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952250 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952253 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952255 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952258 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952261 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952263 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952266 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952269 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952271 2561 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952274 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952276 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952279 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952281 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952283 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952286 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952288 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952291 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952295 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 10:17:03.961483 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952298 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952300 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952303 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952305 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952308 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952310 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952313 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952315 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952318 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952320 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952323 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952326 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952329 2561 feature_gate.go:328] unrecognized feature gate: Example Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952331 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952334 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952337 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952339 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952342 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952346 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 10:17:03.962010 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952349 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952352 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952355 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952358 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952360 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952363 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952367 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952370 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952373 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952375 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952378 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952380 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952393 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952396 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952398 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952401 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952404 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952406 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952408 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 10:17:03.962485 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.952411 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 10:17:03.962968 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.953165 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 10:17:03.962968 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.961296 2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 08 10:17:03.962968 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.961311 2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 08 10:17:03.962968 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.962233 2561 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 08 10:17:03.962968 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.962242 2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 08 10:17:03.962968 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962297 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 10:17:03.962968 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962303 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 10:17:03.962968 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962308 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 10:17:03.962968 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962312 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 10:17:03.962968 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962315 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 10:17:03.962968 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962318 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 10:17:03.962968 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962321 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 10:17:03.962968 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962323 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 10:17:03.962968 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962328 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962331 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962334 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962338 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962341 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962343 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962346 2561 feature_gate.go:328] unrecognized feature gate: Example Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962349 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962351 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962354 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962357 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962360 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962362 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962365 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962368 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962370 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962373 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962375 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962378 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962381 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 10:17:03.963329 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962383 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962387 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962390 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962394 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962397 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962400 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962403 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962406 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962409 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962412 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962414 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962417 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962419 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962422 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962425 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962427 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962430 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962433 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962435 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 10:17:03.963899 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962439 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962442 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962445 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962448 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962450 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962453 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962455 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962458 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962461 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962464 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962466 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962469 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962471 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962473 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962477 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962480 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962483 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962485 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962488 2561 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962491 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 10:17:03.964396 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962494 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962497 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962500 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962502 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962505 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962508 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962510 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962513 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962515 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962518 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962520 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962523 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962536 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962539 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962541 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962544 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962547 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962549 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962552 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 10:17:03.964901 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962554 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 10:17:03.965372 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962557 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 10:17:03.965372 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962559 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 10:17:03.965372 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962562 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 10:17:03.965372 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962564 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 10:17:03.965372 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962567 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 10:17:03.965372 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962569 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 10:17:03.965372 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962573 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 10:17:03.965372 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962575 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 10:17:03.965372 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962578 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 10:17:03.965372 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962581 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 10:17:03.965372 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962583 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 10:17:03.965372 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962586 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 10:17:03.965372 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.962591 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 10:17:03.965372 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962699 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 10:17:03.965372 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962704 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 10:17:03.965372 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962709 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962712 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962715 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962718 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962721 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962724 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962727 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962730 2561 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962732 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962735 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962738 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962741 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962744 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962746 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962749 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962753 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962756 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962758 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962761 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962763 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 10:17:03.965793 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962766 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962768 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962771 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962774 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962777 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962780 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962782 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962785 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962787 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962790 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962792 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962795 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962798 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962800 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962803 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962805 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962808 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962810 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 10:17:03.966293 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962813 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962817 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962821 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962824 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962828 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962830 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962833 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962836 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962838 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962841 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962843 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962846 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962849 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962851 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962854 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962856 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962859 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962863 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962866 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 10:17:03.966878 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962869 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962872 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962874 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962877 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962880 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962882 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962885 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962887 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962890 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962893 2561 feature_gate.go:328] unrecognized feature gate: Example Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962896 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962898 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962901 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962903 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962906 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962909 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962911 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962913 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962916 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962918 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 10:17:03.967345 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962921 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962923 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962926 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962928 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962931 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962933 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962936 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962938 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962940 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962943 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962946 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962949 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962951 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962954 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962956 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962959 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962962 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962964 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962967 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 10:17:03.967838 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:03.962969 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 10:17:03.968305 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.962975 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 10:17:03.968305 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.963083 2561 server.go:962] "Client rotation is on, will bootstrap in background" Sep 08 10:17:03.968305 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.965596 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 08 10:17:03.968305 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.966713 2561 server.go:1019] "Starting client certificate rotation" Sep 08 10:17:03.968305 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.966820 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 10:17:03.968305 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.966854 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 10:17:03.969574 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.969559 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 10:17:03.973420 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.973400 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 10:17:03.984791 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.984766 2561 log.go:25] "Validated CRI v1 runtime API" Sep 08 10:17:03.988083 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.988064 2561 log.go:25] "Validated CRI v1 image API" Sep 08 10:17:03.989827 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.989808 2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 08 10:17:03.993380 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.993359 2561 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b41cdff2-b9a9-4897-944c-1cd295d9b183:/dev/nvme0n1p3 fab0b6f3-23f1-438d-b3ef-8daf6b2356d2:/dev/nvme0n1p4] Sep 08 10:17:03.993439 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.993380 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 08 10:17:03.998672 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.998547 2561 manager.go:216] Machine: {Timestamp:2026-09-08 10:17:03.997031714 +0000 UTC m=+0.366297001 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3109049 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec242640b89c967639d876af1803cec1 SystemUUID:ec242640-b89c-9676-39d8-76af1803cec1 BootID:b81e6a1d-8f3a-41f5-a519-8bf9bfe97aa6 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cf:95:7e:80:17 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cf:95:7e:80:17 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:17:92:90:79:51 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 08 10:17:03.998672 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.998662 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 08 10:17:03.998814 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.998782 2561 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 08 10:17:03.999810 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:03.999796 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 10:17:04.002285 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.002254 2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 08 10:17:04.002424 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.002289 2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-203.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 08 10:17:04.003810 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.003799 2561 topology_manager.go:138] "Creating topology manager with none policy" Sep 08 10:17:04.003851 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.003813 2561 container_manager_linux.go:309] "Creating device plugin manager" Sep 08 10:17:04.003851 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.003821 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 10:17:04.004632 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.004622 2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 10:17:04.006616 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.006604 2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 08 10:17:04.012130 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.012118 2561 state_mem.go:36] "Initialized new in-memory state store" Sep 08 10:17:04.012255 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.012245 2561 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 08 10:17:04.012294 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.012276 2561 kubelet.go:486] "Attempting to sync node with API server" Sep 08 10:17:04.012294 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.012286 2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 08 10:17:04.012360 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.012300 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 08 10:17:04.012360 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.012310 2561 kubelet.go:398] "Adding apiserver pod source" Sep 08 10:17:04.012360 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.012319 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 08 10:17:04.012725 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.012713 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 10:17:04.012766 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.012739 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 10:17:04.015181 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.015160 2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 08 10:17:04.018087 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.018070 2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 08 10:17:04.018087 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.018109 2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 08 10:17:04.018371 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.018358 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 08 10:17:04.018435 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.018377 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 08 10:17:04.018435 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.018386 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 08 10:17:04.018435 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.018395 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 08 10:17:04.018435 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.018409 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 08 10:17:04.018435 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.018418 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 08 10:17:04.018914 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.018903 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 08 10:17:04.018963 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.018916 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 08 10:17:04.018963 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.018933 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 08 10:17:04.018963 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.018942 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 08 10:17:04.018963 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.018955 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 08 10:17:04.019119 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.018968 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 08 10:17:04.019119 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.019001 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 08 10:17:04.019119 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.019010 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 08 10:17:04.022613 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.022593 2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-203.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 08 10:17:04.023165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.023140 2561 server.go:1268] "Started kubelet" Sep 08 10:17:04.023240 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.023159 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 08 10:17:04.023405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.023372 2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 08 10:17:04.023522 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.023483 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 08 10:17:04.023582 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.023507 2561 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 08 10:17:04.023582 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.023440 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-203.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 08 10:17:04.024105 ip-10-0-132-203 systemd[1]: Started Kubernetes Kubelet. Sep 08 10:17:04.028343 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.028314 2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 08 10:17:04.030002 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.029978 2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 08 10:17:04.030576 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.030556 2561 server.go:310] "Adding debug handlers to kubelet server" Sep 08 10:17:04.034785 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.034765 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 08 10:17:04.035387 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.034251 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-203.ec2.internal.18d3511428de75c1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-203.ec2.internal,UID:ip-10-0-132-203.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-203.ec2.internal,},FirstTimestamp:2026-09-08 10:17:04.023082433 +0000 UTC m=+0.392347721,LastTimestamp:2026-09-08 10:17:04.023082433 +0000 UTC m=+0.392347721,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-203.ec2.internal,}" Sep 08 10:17:04.035504 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.035485 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 08 10:17:04.035688 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.035654 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-203.ec2.internal\" not found" Sep 08 10:17:04.036443 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.036426 2561 factory.go:55] Registering systemd factory Sep 08 10:17:04.036518 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.036456 2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 08 10:17:04.036518 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.036457 2561 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 08 10:17:04.036518 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.036483 2561 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 08 10:17:04.036518 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.036461 2561 factory.go:223] Registration of the systemd container factory successfully Sep 08 10:17:04.036705 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.036595 2561 reconstruct.go:97] "Volume reconstruction finished" Sep 08 10:17:04.036705 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.036605 2561 reconciler.go:29] "Reconciler: start to sync state" Sep 08 10:17:04.036774 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.036731 2561 factory.go:153] Registering CRI-O factory Sep 08 10:17:04.036774 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.036744 2561 factory.go:223] Registration of the crio container factory successfully Sep 08 10:17:04.036829 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.036788 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 08 10:17:04.036829 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.036808 2561 factory.go:103] Registering Raw factory Sep 08 10:17:04.036829 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.036825 2561 manager.go:1196] Started watching for new ooms in manager Sep 08 10:17:04.037255 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.037240 2561 manager.go:349] Starting recovery of all containers Sep 08 10:17:04.043070 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.042603 2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-203.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 08 10:17:04.043070 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.042761 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 08 10:17:04.047111 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.047091 2561 manager.go:354] Recovery completed Sep 08 10:17:04.052854 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.052842 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 10:17:04.056734 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.056715 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasSufficientMemory" Sep 08 10:17:04.056810 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.056748 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 10:17:04.056810 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.056758 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasSufficientPID" Sep 08 10:17:04.057396 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.057385 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 10:17:04.057466 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.057389 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 08 10:17:04.057466 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.057417 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 08 10:17:04.057466 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.057433 2561 state_mem.go:36] "Initialized new in-memory state store" Sep 08 10:17:04.058587 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.058456 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-203.ec2.internal.18d351142adfee1c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-203.ec2.internal,UID:ip-10-0-132-203.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-203.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-203.ec2.internal,},FirstTimestamp:2026-09-08 10:17:04.056733212 +0000 UTC m=+0.425998503,LastTimestamp:2026-09-08 10:17:04.056733212 +0000 UTC m=+0.425998503,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-203.ec2.internal,}" Sep 08 10:17:04.059148 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.059131 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasSufficientMemory" Sep 08 10:17:04.059229 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.059157 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 10:17:04.059229 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.059173 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasSufficientPID" Sep 08 10:17:04.059614 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.059600 2561 policy_none.go:49] "None policy: Start" Sep 08 10:17:04.059677 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.059619 2561 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 08 10:17:04.059677 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.059632 2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 08 10:17:04.061557 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.061545 2561 policy_none.go:47] "Start" Sep 08 10:17:04.062707 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.062688 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h7dbl" Sep 08 10:17:04.067242 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.067158 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-203.ec2.internal.18d351142ae03a0e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-203.ec2.internal,UID:ip-10-0-132-203.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-203.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-203.ec2.internal,},FirstTimestamp:2026-09-08 10:17:04.056752654 +0000 UTC m=+0.426017942,LastTimestamp:2026-09-08 10:17:04.056752654 +0000 UTC m=+0.426017942,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-203.ec2.internal,}" Sep 08 10:17:04.071676 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.071650 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h7dbl" Sep 08 10:17:04.097138 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.097122 2561 manager.go:337] "Starting Device Plugin manager" Sep 08 10:17:04.111111 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.097160 2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 08 10:17:04.111111 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.097170 2561 server.go:87] "Starting device plugin registration server" Sep 08 10:17:04.111111 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.097434 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 08 10:17:04.111111 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.097446 2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 08 10:17:04.111111 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.097569 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 08 10:17:04.111111 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.097646 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 08 10:17:04.111111 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.097653 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 08 10:17:04.111111 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.098230 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 08 10:17:04.111111 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.098272 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-203.ec2.internal\" not found" Sep 08 10:17:04.137739 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.137706 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 08 10:17:04.139076 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.139045 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 08 10:17:04.139196 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.139095 2561 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 08 10:17:04.139196 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.139119 2561 kubelet.go:2448] "Starting kubelet main sync loop" Sep 08 10:17:04.139196 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.139157 2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 08 10:17:04.141812 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.141792 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 10:17:04.198173 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.198058 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 10:17:04.199173 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.199156 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasSufficientMemory" Sep 08 10:17:04.199291 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.199192 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 10:17:04.199291 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.199210 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasSufficientPID" Sep 08 10:17:04.199291 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.199238 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.210905 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.210887 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.211015 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.210910 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-203.ec2.internal\": node \"ip-10-0-132-203.ec2.internal\" not found" Sep 08 10:17:04.239762 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.239730 2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-203.ec2.internal"] Sep 08 10:17:04.239877 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.239824 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 10:17:04.240808 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.240789 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasSufficientMemory" Sep 08 10:17:04.240924 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.240820 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 10:17:04.240924 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.240830 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasSufficientPID" Sep 08 10:17:04.242683 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.242662 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-203.ec2.internal\" not found" Sep 08 10:17:04.243280 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.243269 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 10:17:04.243420 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.243407 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.243457 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.243436 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 10:17:04.244237 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.244223 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasSufficientMemory" Sep 08 10:17:04.244307 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.244226 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasSufficientMemory" Sep 08 10:17:04.244307 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.244270 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 10:17:04.244307 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.244282 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasSufficientPID" Sep 08 10:17:04.244307 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.244251 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 10:17:04.244436 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.244315 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasSufficientPID" Sep 08 10:17:04.246562 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.246546 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.246629 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.246575 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 10:17:04.247380 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.247368 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasSufficientMemory" Sep 08 10:17:04.247445 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.247387 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 10:17:04.247445 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.247396 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeHasSufficientPID" Sep 08 10:17:04.262102 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.262078 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-203.ec2.internal\" not found" node="ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.265875 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.265859 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-203.ec2.internal\" not found" node="ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.337666 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.337636 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5ff941279daf8a3e1f7d0667779a0142-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal\" (UID: \"5ff941279daf8a3e1f7d0667779a0142\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.337811 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.337670 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ff941279daf8a3e1f7d0667779a0142-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal\" (UID: \"5ff941279daf8a3e1f7d0667779a0142\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.337811 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.337692 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f4f934834e3ed193d36ec5ddfb5f38fb-config\") pod \"kube-apiserver-proxy-ip-10-0-132-203.ec2.internal\" (UID: \"f4f934834e3ed193d36ec5ddfb5f38fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.343665 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.343645 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-203.ec2.internal\" not found" Sep 08 10:17:04.438382 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.438344 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5ff941279daf8a3e1f7d0667779a0142-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal\" (UID: \"5ff941279daf8a3e1f7d0667779a0142\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.438382 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.438379 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ff941279daf8a3e1f7d0667779a0142-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal\" (UID: \"5ff941279daf8a3e1f7d0667779a0142\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.438603 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.438395 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f4f934834e3ed193d36ec5ddfb5f38fb-config\") pod \"kube-apiserver-proxy-ip-10-0-132-203.ec2.internal\" (UID: \"f4f934834e3ed193d36ec5ddfb5f38fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.438603 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.438452 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5ff941279daf8a3e1f7d0667779a0142-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal\" (UID: \"5ff941279daf8a3e1f7d0667779a0142\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.438603 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.438502 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f4f934834e3ed193d36ec5ddfb5f38fb-config\") pod \"kube-apiserver-proxy-ip-10-0-132-203.ec2.internal\" (UID: \"f4f934834e3ed193d36ec5ddfb5f38fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.438603 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.438549 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ff941279daf8a3e1f7d0667779a0142-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal\" (UID: \"5ff941279daf8a3e1f7d0667779a0142\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.444461 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.444444 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-203.ec2.internal\" not found" Sep 08 10:17:04.545327 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.545248 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-203.ec2.internal\" not found" Sep 08 10:17:04.565427 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.565403 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.568936 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.568918 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.646507 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.646325 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-203.ec2.internal\" not found" Sep 08 10:17:04.747003 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.746983 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-203.ec2.internal\" not found" Sep 08 10:17:04.847313 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.847234 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-203.ec2.internal\" not found" Sep 08 10:17:04.849613 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.849589 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 10:17:04.901999 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.901977 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 10:17:04.937628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.937596 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.954978 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.954960 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 10:17:04.956674 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.956657 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-203.ec2.internal" Sep 08 10:17:04.967047 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.967028 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 08 10:17:04.967193 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.967170 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 10:17:04.967245 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.967171 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 10:17:04.967245 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:04.967183 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 10:17:04.967313 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:04.967253 2561 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a9eb95f534c01429ab70a432bc5f7291-9470c9b2ab032f16.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.132.203:46806->184.192.242.91:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-203.ec2.internal" Sep 08 10:17:05.013874 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.013821 2561 apiserver.go:52] "Watching apiserver" Sep 08 10:17:05.030509 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.030480 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 08 10:17:05.031624 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.031569 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-xlksv","openshift-network-operator/iptables-alerter-wkwnr","openshift-ovn-kubernetes/ovnkube-node-wx64r","kube-system/global-pull-secret-syncer-56snn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8","openshift-dns/node-resolver-lw9vz","openshift-image-registry/node-ca-pv25g","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal","openshift-multus/multus-4mxks","openshift-multus/multus-additional-cni-plugins-552hj","openshift-multus/network-metrics-daemon-pr8g9","kube-system/konnectivity-agent-qhxw7","kube-system/kube-apiserver-proxy-ip-10-0-132-203.ec2.internal","openshift-cluster-node-tuning-operator/tuned-lsnsb"] Sep 08 10:17:05.034316 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.034301 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:05.034401 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.034375 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:05.035647 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.035632 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 08 10:17:05.036614 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.036590 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wkwnr" Sep 08 10:17:05.039091 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.039071 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.042802 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.042109 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:05.042802 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.042187 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 08 10:17:05.042802 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.042394 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 08 10:17:05.042802 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.042454 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:05.043089 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.042851 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-cni-bin\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.043089 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.042894 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-ovnkube-config\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.043089 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.042962 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6l9t\" (UniqueName: \"kubernetes.io/projected/4425cf1f-0af5-4990-9169-4ce1848e739d-kube-api-access-c6l9t\") pod \"iptables-alerter-wkwnr\" (UID: \"4425cf1f-0af5-4990-9169-4ce1848e739d\") " pod="openshift-network-operator/iptables-alerter-wkwnr" Sep 08 10:17:05.043089 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.042988 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-slash\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.043089 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.043017 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-run-ovn\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.043089 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.043067 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-node-log\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.043424 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.043096 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-cni-netd\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.043424 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.043124 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-ovnkube-script-lib\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.043424 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.043249 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqwvr\" (UniqueName: \"kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr\") pod \"network-check-target-xlksv\" (UID: \"0d5943f3-ba9e-4dc9-91c1-9787f83d5305\") " pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:05.043424 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.043127 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 08 10:17:05.043683 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.043651 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-etc-openvswitch\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.043739 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.043689 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-log-socket\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.043788 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.043742 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-run-ovn-kubernetes\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.043892 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.043784 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 08 10:17:05.043892 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.043815 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-env-overrides\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.043892 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.043854 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4425cf1f-0af5-4990-9169-4ce1848e739d-host-slash\") pod \"iptables-alerter-wkwnr\" (UID: \"4425cf1f-0af5-4990-9169-4ce1848e739d\") " pod="openshift-network-operator/iptables-alerter-wkwnr" Sep 08 10:17:05.044037 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.043941 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-systemd-units\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.044037 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.044004 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-run-netns\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.044142 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.044116 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.044206 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.044184 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-ovn-node-metrics-cert\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.044263 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.044224 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5bmv\" (UniqueName: \"kubernetes.io/projected/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-kube-api-access-x5bmv\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.044263 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.044257 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4425cf1f-0af5-4990-9169-4ce1848e739d-iptables-alerter-script\") pod \"iptables-alerter-wkwnr\" (UID: \"4425cf1f-0af5-4990-9169-4ce1848e739d\") " pod="openshift-network-operator/iptables-alerter-wkwnr" Sep 08 10:17:05.044372 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.044271 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wxclr\"" Sep 08 10:17:05.044372 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.044344 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 08 10:17:05.044372 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.044356 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 08 10:17:05.044518 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.044367 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-kubelet\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.044518 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.044278 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 08 10:17:05.044518 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.044408 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-run-systemd\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.044700 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.044576 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-var-lib-openvswitch\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.044700 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.044655 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-run-openvswitch\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.044700 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.044660 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 08 10:17:05.044847 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.044702 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 08 10:17:05.045074 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.044921 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xfn4b\"" Sep 08 10:17:05.046151 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.046133 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.048682 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.048664 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lw9vz" Sep 08 10:17:05.051072 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.051053 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pv25g" Sep 08 10:17:05.052527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.052512 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lr66n\"" Sep 08 10:17:05.053038 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.053023 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 08 10:17:05.053454 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.053441 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.055618 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.055596 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 08 10:17:05.055872 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.055850 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 08 10:17:05.055872 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.055873 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 08 10:17:05.056080 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.055877 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mrrh5\"" Sep 08 10:17:05.056080 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.055963 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 08 10:17:05.056080 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.055989 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 08 10:17:05.056080 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.055992 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 08 10:17:05.056258 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.056092 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 08 10:17:05.056258 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.056180 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 08 10:17:05.056258 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.056247 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.056596 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.056570 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 08 10:17:05.056708 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.056617 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mr95b\"" Sep 08 10:17:05.056708 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.056660 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-txchj\"" Sep 08 10:17:05.056708 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.056621 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 08 10:17:05.056708 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.056698 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 08 10:17:05.058515 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.058488 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:05.058664 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.058586 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:05.058897 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.058882 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 08 10:17:05.058978 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.058964 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 08 10:17:05.059077 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.058967 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fdk2w\"" Sep 08 10:17:05.059276 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.059258 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 10:17:05.060988 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.060966 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qhxw7" Sep 08 10:17:05.063477 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.063447 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 08 10:17:05.063477 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.063460 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 08 10:17:05.063623 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.063576 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.063719 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.063704 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-w4lwh\"" Sep 08 10:17:05.065880 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.065862 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xdchj\"" Sep 08 10:17:05.066980 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.066186 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 08 10:17:05.066980 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.066385 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 08 10:17:05.074415 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.074388 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 10:12:04 +0000 UTC" deadline="2028-03-29 17:49:36.931348739 +0000 UTC" Sep 08 10:17:05.074415 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.074412 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13639h32m31.856939146s" Sep 08 10:17:05.087312 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.087294 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-m7vk8" Sep 08 10:17:05.097014 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.096990 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-m7vk8" Sep 08 10:17:05.138567 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.138539 2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 08 10:17:05.144900 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.144797 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-run-k8s-cni-cncf-io\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.144900 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.144842 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-ovnkube-script-lib\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.144900 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.144874 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08fd25ee-4cf2-467f-aa66-cb3f454c528c-tmp\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.145103 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.144901 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.145103 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.144931 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnxfk\" (UniqueName: \"kubernetes.io/projected/b72101be-602c-4ea1-85fc-9baade17727b-kube-api-access-cnxfk\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.145103 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.144977 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ee8913a-efe5-42f7-bf88-3a0439a828ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.145103 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.144999 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-dbus\") pod \"global-pull-secret-syncer-56snn\" (UID: \"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2\") " pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:05.145103 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145018 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqwvr\" (UniqueName: \"kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr\") pod \"network-check-target-xlksv\" (UID: \"0d5943f3-ba9e-4dc9-91c1-9787f83d5305\") " pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:05.145103 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145042 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-etc-openvswitch\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.145103 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145066 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-log-socket\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.145103 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145091 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-run-ovn-kubernetes\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.145448 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145133 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-systemd\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.145448 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145134 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-etc-openvswitch\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.145448 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145157 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs\") pod \"network-metrics-daemon-pr8g9\" (UID: \"db4c4c5b-19b6-439a-bcd2-0631f7626479\") " pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:05.145448 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145181 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-ovn-node-metrics-cert\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.145448 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145204 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-run-ovn-kubernetes\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.145448 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145205 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5bmv\" (UniqueName: \"kubernetes.io/projected/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-kube-api-access-x5bmv\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.145448 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145252 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdjhf\" (UniqueName: \"kubernetes.io/projected/08fd25ee-4cf2-467f-aa66-cb3f454c528c-kube-api-access-wdjhf\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.145448 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145284 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-etc-selinux\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.145448 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145311 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-run-netns\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.145448 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145337 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-var-lib-cni-bin\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.145448 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145359 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-hostroot\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.145448 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145381 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-multus-conf-dir\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.145448 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145398 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-log-socket\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.145448 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145406 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-run\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145554 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9f96208-ea87-4822-ae72-9c07e5d5fa8f-tmp-dir\") pod \"node-resolver-lw9vz\" (UID: \"e9f96208-ea87-4822-ae72-9c07e5d5fa8f\") " pod="openshift-dns/node-resolver-lw9vz" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145605 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4425cf1f-0af5-4990-9169-4ce1848e739d-iptables-alerter-script\") pod \"iptables-alerter-wkwnr\" (UID: \"4425cf1f-0af5-4990-9169-4ce1848e739d\") " pod="openshift-network-operator/iptables-alerter-wkwnr" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145634 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-var-lib-openvswitch\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145660 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-ovnkube-config\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145686 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-etc-kubernetes\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145684 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-var-lib-openvswitch\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145713 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ee8913a-efe5-42f7-bf88-3a0439a828ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145747 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-run-ovn\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145773 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-var-lib-kubelet\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145695 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145801 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ee8913a-efe5-42f7-bf88-3a0439a828ba-system-cni-dir\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145827 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ee8913a-efe5-42f7-bf88-3a0439a828ba-cnibin\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145838 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-run-ovn\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145853 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ee8913a-efe5-42f7-bf88-3a0439a828ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145876 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/919f3306-bc79-43be-92f0-2b46f301d6f7-host\") pod \"node-ca-pv25g\" (UID: \"919f3306-bc79-43be-92f0-2b46f301d6f7\") " pod="openshift-image-registry/node-ca-pv25g" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145913 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn66m\" (UniqueName: \"kubernetes.io/projected/db4c4c5b-19b6-439a-bcd2-0631f7626479-kube-api-access-tn66m\") pod \"network-metrics-daemon-pr8g9\" (UID: \"db4c4c5b-19b6-439a-bcd2-0631f7626479\") " pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:05.146143 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145944 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a910ad03-91d3-448e-bf66-56effa2459f3-multus-daemon-config\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145972 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-systemd-units\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.145990 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-run-netns\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146009 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a910ad03-91d3-448e-bf66-56effa2459f3-cni-binary-copy\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146034 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/63e77b90-9f35-4b51-a977-290598c60b83-konnectivity-ca\") pod \"konnectivity-agent-qhxw7\" (UID: \"63e77b90-9f35-4b51-a977-290598c60b83\") " pod="kube-system/konnectivity-agent-qhxw7" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146048 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-systemd-units\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146059 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-lib-modules\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146062 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-run-netns\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146098 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-host\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146123 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e9f96208-ea87-4822-ae72-9c07e5d5fa8f-hosts-file\") pod \"node-resolver-lw9vz\" (UID: \"e9f96208-ea87-4822-ae72-9c07e5d5fa8f\") " pod="openshift-dns/node-resolver-lw9vz" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146146 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-system-cni-dir\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146156 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-ovnkube-config\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146182 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4425cf1f-0af5-4990-9169-4ce1848e739d-iptables-alerter-script\") pod \"iptables-alerter-wkwnr\" (UID: \"4425cf1f-0af5-4990-9169-4ce1848e739d\") " pod="openshift-network-operator/iptables-alerter-wkwnr" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146173 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-slash\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146221 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-slash\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146247 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-run-multus-certs\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146276 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r7d5\" (UniqueName: \"kubernetes.io/projected/6ee8913a-efe5-42f7-bf88-3a0439a828ba-kube-api-access-6r7d5\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.147349 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146300 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-socket-dir\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146312 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-ovnkube-script-lib\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146331 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-env-overrides\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146356 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-modprobe-d\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146386 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/919f3306-bc79-43be-92f0-2b46f301d6f7-serviceca\") pod \"node-ca-pv25g\" (UID: \"919f3306-bc79-43be-92f0-2b46f301d6f7\") " pod="openshift-image-registry/node-ca-pv25g" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146411 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-kubelet-config\") pod \"global-pull-secret-syncer-56snn\" (UID: \"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2\") " pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146449 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-sys-fs\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146486 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-multus-socket-dir-parent\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146520 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4425cf1f-0af5-4990-9169-4ce1848e739d-host-slash\") pod \"iptables-alerter-wkwnr\" (UID: \"4425cf1f-0af5-4990-9169-4ce1848e739d\") " pod="openshift-network-operator/iptables-alerter-wkwnr" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146563 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4425cf1f-0af5-4990-9169-4ce1848e739d-host-slash\") pod \"iptables-alerter-wkwnr\" (UID: \"4425cf1f-0af5-4990-9169-4ce1848e739d\") " pod="openshift-network-operator/iptables-alerter-wkwnr" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146565 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ee8913a-efe5-42f7-bf88-3a0439a828ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146608 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-sysctl-conf\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146632 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-kubelet\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146651 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-run-systemd\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146672 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-run-openvswitch\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146693 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-cni-bin\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146699 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-kubelet\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.148165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146707 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-run-systemd\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146715 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-sysctl-d\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146731 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-env-overrides\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146746 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-multus-cni-dir\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146752 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-run-openvswitch\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146775 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6l9t\" (UniqueName: \"kubernetes.io/projected/4425cf1f-0af5-4990-9169-4ce1848e739d-kube-api-access-c6l9t\") pod \"iptables-alerter-wkwnr\" (UID: \"4425cf1f-0af5-4990-9169-4ce1848e739d\") " pod="openshift-network-operator/iptables-alerter-wkwnr" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146783 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-cni-bin\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146815 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-node-log\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146832 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-cni-netd\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146879 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-cni-netd\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146886 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-node-log\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146881 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-kubernetes\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146918 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-sys\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146943 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-sysconfig\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146964 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-tuned\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.146987 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6g4x\" (UniqueName: \"kubernetes.io/projected/e9f96208-ea87-4822-ae72-9c07e5d5fa8f-kube-api-access-s6g4x\") pod \"node-resolver-lw9vz\" (UID: \"e9f96208-ea87-4822-ae72-9c07e5d5fa8f\") " pod="openshift-dns/node-resolver-lw9vz" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.147021 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mk7x\" (UniqueName: \"kubernetes.io/projected/919f3306-bc79-43be-92f0-2b46f301d6f7-kube-api-access-2mk7x\") pod \"node-ca-pv25g\" (UID: \"919f3306-bc79-43be-92f0-2b46f301d6f7\") " pod="openshift-image-registry/node-ca-pv25g" Sep 08 10:17:05.148906 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.147052 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-var-lib-cni-multus\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.149712 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.147079 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgt8n\" (UniqueName: \"kubernetes.io/projected/a910ad03-91d3-448e-bf66-56effa2459f3-kube-api-access-pgt8n\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.149712 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.147105 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/63e77b90-9f35-4b51-a977-290598c60b83-agent-certs\") pod \"konnectivity-agent-qhxw7\" (UID: \"63e77b90-9f35-4b51-a977-290598c60b83\") " pod="kube-system/konnectivity-agent-qhxw7" Sep 08 10:17:05.149712 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.147141 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.149712 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.147172 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-registration-dir\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.149712 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.147204 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.149712 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.147209 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-cnibin\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.149712 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.147244 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-var-lib-kubelet\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.149712 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.147260 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret\") pod \"global-pull-secret-syncer-56snn\" (UID: \"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2\") " pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:05.149712 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.147295 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-device-dir\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.149712 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.149671 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-ovn-node-metrics-cert\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.153309 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.153289 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 10:17:05.153309 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.153310 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 10:17:05.153447 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.153320 2561 projected.go:196] Error preparing data for projected volume kube-api-access-hqwvr for pod openshift-network-diagnostics/network-check-target-xlksv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:17:05.153447 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.153380 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr podName:0d5943f3-ba9e-4dc9-91c1-9787f83d5305 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:05.653364755 +0000 UTC m=+2.022630031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hqwvr" (UniqueName: "kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr") pod "network-check-target-xlksv" (UID: "0d5943f3-ba9e-4dc9-91c1-9787f83d5305") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:17:05.155521 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.155501 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6l9t\" (UniqueName: \"kubernetes.io/projected/4425cf1f-0af5-4990-9169-4ce1848e739d-kube-api-access-c6l9t\") pod \"iptables-alerter-wkwnr\" (UID: \"4425cf1f-0af5-4990-9169-4ce1848e739d\") " pod="openshift-network-operator/iptables-alerter-wkwnr" Sep 08 10:17:05.155758 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.155740 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5bmv\" (UniqueName: \"kubernetes.io/projected/d7f1ba8a-c265-4a99-ae50-00c5b2b3c217-kube-api-access-x5bmv\") pod \"ovnkube-node-wx64r\" (UID: \"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217\") " pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.248546 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248503 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-registration-dir\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.248798 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248559 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-cnibin\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.248798 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248576 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-var-lib-kubelet\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.248798 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248644 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-registration-dir\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.248798 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248646 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-cnibin\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.248798 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248646 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret\") pod \"global-pull-secret-syncer-56snn\" (UID: \"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2\") " pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:05.248798 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248693 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-var-lib-kubelet\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.248798 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248695 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-device-dir\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.248798 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248717 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-run-k8s-cni-cncf-io\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.248798 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248728 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-device-dir\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.248798 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248733 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08fd25ee-4cf2-467f-aa66-cb3f454c528c-tmp\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.248798 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.248741 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 10:17:05.248798 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248752 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.248798 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248756 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-run-k8s-cni-cncf-io\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.248798 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248789 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.249548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248806 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnxfk\" (UniqueName: \"kubernetes.io/projected/b72101be-602c-4ea1-85fc-9baade17727b-kube-api-access-cnxfk\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.249548 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.248817 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret podName:e0ef2303-d3cc-4560-b248-8d0c1b6e94e2 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:05.748795785 +0000 UTC m=+2.118061067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret") pod "global-pull-secret-syncer-56snn" (UID: "e0ef2303-d3cc-4560-b248-8d0c1b6e94e2") : object "kube-system"/"original-pull-secret" not registered Sep 08 10:17:05.249548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248894 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ee8913a-efe5-42f7-bf88-3a0439a828ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.249548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248912 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-dbus\") pod \"global-pull-secret-syncer-56snn\" (UID: \"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2\") " pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:05.249548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248941 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-systemd\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.249548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248956 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs\") pod \"network-metrics-daemon-pr8g9\" (UID: \"db4c4c5b-19b6-439a-bcd2-0631f7626479\") " pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:05.249548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248974 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdjhf\" (UniqueName: \"kubernetes.io/projected/08fd25ee-4cf2-467f-aa66-cb3f454c528c-kube-api-access-wdjhf\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.249548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.248997 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-etc-selinux\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.249548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249021 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-run-netns\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.249548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249047 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-systemd\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.249548 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.249058 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:17:05.249548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249093 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-dbus\") pod \"global-pull-secret-syncer-56snn\" (UID: \"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2\") " pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:05.249548 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.249109 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs podName:db4c4c5b-19b6-439a-bcd2-0631f7626479 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:05.749093588 +0000 UTC m=+2.118358865 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs") pod "network-metrics-daemon-pr8g9" (UID: "db4c4c5b-19b6-439a-bcd2-0631f7626479") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:17:05.249548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249100 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-var-lib-cni-bin\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.249548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249118 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-etc-selinux\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.249548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249059 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-var-lib-cni-bin\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249159 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-hostroot\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249183 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-multus-conf-dir\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249208 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-run\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249236 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-hostroot\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249252 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9f96208-ea87-4822-ae72-9c07e5d5fa8f-tmp-dir\") pod \"node-resolver-lw9vz\" (UID: \"e9f96208-ea87-4822-ae72-9c07e5d5fa8f\") " pod="openshift-dns/node-resolver-lw9vz" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249237 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-run-netns\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249274 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-run\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249242 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-multus-conf-dir\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249280 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-etc-kubernetes\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249321 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ee8913a-efe5-42f7-bf88-3a0439a828ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249351 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-var-lib-kubelet\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249343 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-etc-kubernetes\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249395 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ee8913a-efe5-42f7-bf88-3a0439a828ba-system-cni-dir\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249424 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ee8913a-efe5-42f7-bf88-3a0439a828ba-cnibin\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249423 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-var-lib-kubelet\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249444 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ee8913a-efe5-42f7-bf88-3a0439a828ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249468 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/919f3306-bc79-43be-92f0-2b46f301d6f7-host\") pod \"node-ca-pv25g\" (UID: \"919f3306-bc79-43be-92f0-2b46f301d6f7\") " pod="openshift-image-registry/node-ca-pv25g" Sep 08 10:17:05.250406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249493 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ee8913a-efe5-42f7-bf88-3a0439a828ba-cnibin\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249507 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn66m\" (UniqueName: \"kubernetes.io/projected/db4c4c5b-19b6-439a-bcd2-0631f7626479-kube-api-access-tn66m\") pod \"network-metrics-daemon-pr8g9\" (UID: \"db4c4c5b-19b6-439a-bcd2-0631f7626479\") " pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249551 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ee8913a-efe5-42f7-bf88-3a0439a828ba-system-cni-dir\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249570 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ee8913a-efe5-42f7-bf88-3a0439a828ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249587 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a910ad03-91d3-448e-bf66-56effa2459f3-multus-daemon-config\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249597 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/919f3306-bc79-43be-92f0-2b46f301d6f7-host\") pod \"node-ca-pv25g\" (UID: \"919f3306-bc79-43be-92f0-2b46f301d6f7\") " pod="openshift-image-registry/node-ca-pv25g" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249621 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a910ad03-91d3-448e-bf66-56effa2459f3-cni-binary-copy\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249627 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9f96208-ea87-4822-ae72-9c07e5d5fa8f-tmp-dir\") pod \"node-resolver-lw9vz\" (UID: \"e9f96208-ea87-4822-ae72-9c07e5d5fa8f\") " pod="openshift-dns/node-resolver-lw9vz" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249648 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/63e77b90-9f35-4b51-a977-290598c60b83-konnectivity-ca\") pod \"konnectivity-agent-qhxw7\" (UID: \"63e77b90-9f35-4b51-a977-290598c60b83\") " pod="kube-system/konnectivity-agent-qhxw7" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249676 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-lib-modules\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249723 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-host\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249755 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e9f96208-ea87-4822-ae72-9c07e5d5fa8f-hosts-file\") pod \"node-resolver-lw9vz\" (UID: \"e9f96208-ea87-4822-ae72-9c07e5d5fa8f\") " pod="openshift-dns/node-resolver-lw9vz" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249774 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-lib-modules\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249780 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-system-cni-dir\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249812 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-run-multus-certs\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249839 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r7d5\" (UniqueName: \"kubernetes.io/projected/6ee8913a-efe5-42f7-bf88-3a0439a828ba-kube-api-access-6r7d5\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249870 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-socket-dir\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249901 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-modprobe-d\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.251342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249925 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/919f3306-bc79-43be-92f0-2b46f301d6f7-serviceca\") pod \"node-ca-pv25g\" (UID: \"919f3306-bc79-43be-92f0-2b46f301d6f7\") " pod="openshift-image-registry/node-ca-pv25g" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249953 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-kubelet-config\") pod \"global-pull-secret-syncer-56snn\" (UID: \"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2\") " pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.249978 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-sys-fs\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250004 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-multus-socket-dir-parent\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250299 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a910ad03-91d3-448e-bf66-56effa2459f3-cni-binary-copy\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250365 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-socket-dir\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250377 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/63e77b90-9f35-4b51-a977-290598c60b83-konnectivity-ca\") pod \"konnectivity-agent-qhxw7\" (UID: \"63e77b90-9f35-4b51-a977-290598c60b83\") " pod="kube-system/konnectivity-agent-qhxw7" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250370 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-kubelet-config\") pod \"global-pull-secret-syncer-56snn\" (UID: \"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2\") " pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250383 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ee8913a-efe5-42f7-bf88-3a0439a828ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250418 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ee8913a-efe5-42f7-bf88-3a0439a828ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250448 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b72101be-602c-4ea1-85fc-9baade17727b-sys-fs\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250458 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-sysctl-conf\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250496 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-sysctl-d\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250493 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-run-multus-certs\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250524 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-multus-cni-dir\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250552 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-kubernetes\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250559 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a910ad03-91d3-448e-bf66-56effa2459f3-multus-daemon-config\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.252000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250572 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-sys\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250592 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-sysconfig\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250631 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-tuned\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250646 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-sysctl-conf\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250659 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6g4x\" (UniqueName: \"kubernetes.io/projected/e9f96208-ea87-4822-ae72-9c07e5d5fa8f-kube-api-access-s6g4x\") pod \"node-resolver-lw9vz\" (UID: \"e9f96208-ea87-4822-ae72-9c07e5d5fa8f\") " pod="openshift-dns/node-resolver-lw9vz" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250686 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mk7x\" (UniqueName: \"kubernetes.io/projected/919f3306-bc79-43be-92f0-2b46f301d6f7-kube-api-access-2mk7x\") pod \"node-ca-pv25g\" (UID: \"919f3306-bc79-43be-92f0-2b46f301d6f7\") " pod="openshift-image-registry/node-ca-pv25g" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250703 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-host\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250708 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-var-lib-cni-multus\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250738 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-kubernetes\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250747 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-host-var-lib-cni-multus\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250752 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgt8n\" (UniqueName: \"kubernetes.io/projected/a910ad03-91d3-448e-bf66-56effa2459f3-kube-api-access-pgt8n\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250798 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-sys\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250819 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/919f3306-bc79-43be-92f0-2b46f301d6f7-serviceca\") pod \"node-ca-pv25g\" (UID: \"919f3306-bc79-43be-92f0-2b46f301d6f7\") " pod="openshift-image-registry/node-ca-pv25g" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250827 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/63e77b90-9f35-4b51-a977-290598c60b83-agent-certs\") pod \"konnectivity-agent-qhxw7\" (UID: \"63e77b90-9f35-4b51-a977-290598c60b83\") " pod="kube-system/konnectivity-agent-qhxw7" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250857 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ee8913a-efe5-42f7-bf88-3a0439a828ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250830 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-sysctl-d\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250899 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-sysconfig\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250916 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-modprobe-d\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.252527 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.250952 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e9f96208-ea87-4822-ae72-9c07e5d5fa8f-hosts-file\") pod \"node-resolver-lw9vz\" (UID: \"e9f96208-ea87-4822-ae72-9c07e5d5fa8f\") " pod="openshift-dns/node-resolver-lw9vz" Sep 08 10:17:05.253226 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.251016 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ee8913a-efe5-42f7-bf88-3a0439a828ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.253226 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.251018 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-multus-cni-dir\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.253226 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.251082 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-system-cni-dir\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.253226 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.251192 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a910ad03-91d3-448e-bf66-56effa2459f3-multus-socket-dir-parent\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.253226 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.251621 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08fd25ee-4cf2-467f-aa66-cb3f454c528c-tmp\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.253226 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.252867 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08fd25ee-4cf2-467f-aa66-cb3f454c528c-etc-tuned\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.253436 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.253348 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/63e77b90-9f35-4b51-a977-290598c60b83-agent-certs\") pod \"konnectivity-agent-qhxw7\" (UID: \"63e77b90-9f35-4b51-a977-290598c60b83\") " pod="kube-system/konnectivity-agent-qhxw7" Sep 08 10:17:05.258959 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.258935 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnxfk\" (UniqueName: \"kubernetes.io/projected/b72101be-602c-4ea1-85fc-9baade17727b-kube-api-access-cnxfk\") pod \"aws-ebs-csi-driver-node-7pjw8\" (UID: \"b72101be-602c-4ea1-85fc-9baade17727b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.259738 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.259722 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdjhf\" (UniqueName: \"kubernetes.io/projected/08fd25ee-4cf2-467f-aa66-cb3f454c528c-kube-api-access-wdjhf\") pod \"tuned-lsnsb\" (UID: \"08fd25ee-4cf2-467f-aa66-cb3f454c528c\") " pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.263321 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.263288 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgt8n\" (UniqueName: \"kubernetes.io/projected/a910ad03-91d3-448e-bf66-56effa2459f3-kube-api-access-pgt8n\") pod \"multus-4mxks\" (UID: \"a910ad03-91d3-448e-bf66-56effa2459f3\") " pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.263437 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.263366 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn66m\" (UniqueName: \"kubernetes.io/projected/db4c4c5b-19b6-439a-bcd2-0631f7626479-kube-api-access-tn66m\") pod \"network-metrics-daemon-pr8g9\" (UID: \"db4c4c5b-19b6-439a-bcd2-0631f7626479\") " pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:05.263672 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.263654 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mk7x\" (UniqueName: \"kubernetes.io/projected/919f3306-bc79-43be-92f0-2b46f301d6f7-kube-api-access-2mk7x\") pod \"node-ca-pv25g\" (UID: \"919f3306-bc79-43be-92f0-2b46f301d6f7\") " pod="openshift-image-registry/node-ca-pv25g" Sep 08 10:17:05.263929 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.263912 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r7d5\" (UniqueName: \"kubernetes.io/projected/6ee8913a-efe5-42f7-bf88-3a0439a828ba-kube-api-access-6r7d5\") pod \"multus-additional-cni-plugins-552hj\" (UID: \"6ee8913a-efe5-42f7-bf88-3a0439a828ba\") " pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.264946 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.264925 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6g4x\" (UniqueName: \"kubernetes.io/projected/e9f96208-ea87-4822-ae72-9c07e5d5fa8f-kube-api-access-s6g4x\") pod \"node-resolver-lw9vz\" (UID: \"e9f96208-ea87-4822-ae72-9c07e5d5fa8f\") " pod="openshift-dns/node-resolver-lw9vz" Sep 08 10:17:05.302679 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.302645 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 10:17:05.324433 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:05.324405 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ff941279daf8a3e1f7d0667779a0142.slice/crio-f2d63f2fdde295f4931db473fbfbed14595b354a4a885204c5f245220b826381 WatchSource:0}: Error finding container f2d63f2fdde295f4931db473fbfbed14595b354a4a885204c5f245220b826381: Status 404 returned error can't find the container with id f2d63f2fdde295f4931db473fbfbed14595b354a4a885204c5f245220b826381 Sep 08 10:17:05.325421 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:05.325403 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f934834e3ed193d36ec5ddfb5f38fb.slice/crio-a465d55ba4ff7ea592cda6f2b25b9f05e7ed39777af48687b083dc3c926bdde5 WatchSource:0}: Error finding container a465d55ba4ff7ea592cda6f2b25b9f05e7ed39777af48687b083dc3c926bdde5: Status 404 returned error can't find the container with id a465d55ba4ff7ea592cda6f2b25b9f05e7ed39777af48687b083dc3c926bdde5 Sep 08 10:17:05.329056 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.329040 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 10:17:05.363250 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.363164 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wkwnr" Sep 08 10:17:05.373486 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:05.373457 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4425cf1f_0af5_4990_9169_4ce1848e739d.slice/crio-aaab16d59cebd334304d7f9a285636b8c88f269adc27ceb66d6ea578879dbf10 WatchSource:0}: Error finding container aaab16d59cebd334304d7f9a285636b8c88f269adc27ceb66d6ea578879dbf10: Status 404 returned error can't find the container with id aaab16d59cebd334304d7f9a285636b8c88f269adc27ceb66d6ea578879dbf10 Sep 08 10:17:05.382040 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.382021 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:05.390730 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:05.390709 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7f1ba8a_c265_4a99_ae50_00c5b2b3c217.slice/crio-c5ff6b9c995a4e5cbbe4dec54e4738e9332aa58b9e31553c7e1efae744ade81b WatchSource:0}: Error finding container c5ff6b9c995a4e5cbbe4dec54e4738e9332aa58b9e31553c7e1efae744ade81b: Status 404 returned error can't find the container with id c5ff6b9c995a4e5cbbe4dec54e4738e9332aa58b9e31553c7e1efae744ade81b Sep 08 10:17:05.400422 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.400404 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" Sep 08 10:17:05.409162 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:05.409129 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb72101be_602c_4ea1_85fc_9baade17727b.slice/crio-e79953798864ab85522344798919f34b2914183dd43e10eecc1fd021c86a6f24 WatchSource:0}: Error finding container e79953798864ab85522344798919f34b2914183dd43e10eecc1fd021c86a6f24: Status 404 returned error can't find the container with id e79953798864ab85522344798919f34b2914183dd43e10eecc1fd021c86a6f24 Sep 08 10:17:05.413335 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.413318 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lw9vz" Sep 08 10:17:05.421394 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:05.421374 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9f96208_ea87_4822_ae72_9c07e5d5fa8f.slice/crio-d5aaea3d7fe4eb18e773b95e52d11893b2b21a6ffd1c7728b098537a08f89a1b WatchSource:0}: Error finding container d5aaea3d7fe4eb18e773b95e52d11893b2b21a6ffd1c7728b098537a08f89a1b: Status 404 returned error can't find the container with id d5aaea3d7fe4eb18e773b95e52d11893b2b21a6ffd1c7728b098537a08f89a1b Sep 08 10:17:05.426479 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.426450 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pv25g" Sep 08 10:17:05.430402 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.430382 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4mxks" Sep 08 10:17:05.434834 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:05.434812 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod919f3306_bc79_43be_92f0_2b46f301d6f7.slice/crio-cd8278a69d743a632354d3ce9e5876fe2d25cde039746022803832168fc3ea06 WatchSource:0}: Error finding container cd8278a69d743a632354d3ce9e5876fe2d25cde039746022803832168fc3ea06: Status 404 returned error can't find the container with id cd8278a69d743a632354d3ce9e5876fe2d25cde039746022803832168fc3ea06 Sep 08 10:17:05.441747 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.441728 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-552hj" Sep 08 10:17:05.448786 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.448768 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qhxw7" Sep 08 10:17:05.450762 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:05.450740 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ee8913a_efe5_42f7_bf88_3a0439a828ba.slice/crio-eae40cab6d2c3cfa46c38c8054c80d3e5b7d1e471d3dd89ed9da0c297490cf17 WatchSource:0}: Error finding container eae40cab6d2c3cfa46c38c8054c80d3e5b7d1e471d3dd89ed9da0c297490cf17: Status 404 returned error can't find the container with id eae40cab6d2c3cfa46c38c8054c80d3e5b7d1e471d3dd89ed9da0c297490cf17 Sep 08 10:17:05.457183 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:05.457157 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63e77b90_9f35_4b51_a977_290598c60b83.slice/crio-1f91f2fe2ad2b871540f8441ed6adbf5de13ab77911389af1d203f986ccaaf4a WatchSource:0}: Error finding container 1f91f2fe2ad2b871540f8441ed6adbf5de13ab77911389af1d203f986ccaaf4a: Status 404 returned error can't find the container with id 1f91f2fe2ad2b871540f8441ed6adbf5de13ab77911389af1d203f986ccaaf4a Sep 08 10:17:05.465779 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.465762 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" Sep 08 10:17:05.477015 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:05.476451 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08fd25ee_4cf2_467f_aa66_cb3f454c528c.slice/crio-a3b867b107e13daa364bbce3e3bd8b6cdeb0e99dff78fcfd42e5fbcd3363900a WatchSource:0}: Error finding container a3b867b107e13daa364bbce3e3bd8b6cdeb0e99dff78fcfd42e5fbcd3363900a: Status 404 returned error can't find the container with id a3b867b107e13daa364bbce3e3bd8b6cdeb0e99dff78fcfd42e5fbcd3363900a Sep 08 10:17:05.478593 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.478572 2561 kuberuntime_manager.go:1459] "Unhandled Error" err="container tuned start failed in pod tuned-lsnsb_openshift-cluster-node-tuning-operator(08fd25ee-4cf2-467f-aa66-cb3f454c528c): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Sep 08 10:17:05.478662 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.478628 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"tuned\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" podUID="08fd25ee-4cf2-467f-aa66-cb3f454c528c" Sep 08 10:17:05.653813 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.653707 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqwvr\" (UniqueName: \"kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr\") pod \"network-check-target-xlksv\" (UID: \"0d5943f3-ba9e-4dc9-91c1-9787f83d5305\") " pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:05.653976 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.653879 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 10:17:05.653976 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.653903 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 10:17:05.653976 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.653914 2561 projected.go:196] Error preparing data for projected volume kube-api-access-hqwvr for pod openshift-network-diagnostics/network-check-target-xlksv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:17:05.653976 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.653969 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr podName:0d5943f3-ba9e-4dc9-91c1-9787f83d5305 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:06.653954391 +0000 UTC m=+3.023219686 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hqwvr" (UniqueName: "kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr") pod "network-check-target-xlksv" (UID: "0d5943f3-ba9e-4dc9-91c1-9787f83d5305") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:17:05.754826 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.754785 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs\") pod \"network-metrics-daemon-pr8g9\" (UID: \"db4c4c5b-19b6-439a-bcd2-0631f7626479\") " pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:05.755006 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.754852 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret\") pod \"global-pull-secret-syncer-56snn\" (UID: \"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2\") " pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:05.755006 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.754953 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:17:05.755006 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.754971 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 10:17:05.755111 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.755017 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret podName:e0ef2303-d3cc-4560-b248-8d0c1b6e94e2 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:06.7550042 +0000 UTC m=+3.124269477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret") pod "global-pull-secret-syncer-56snn" (UID: "e0ef2303-d3cc-4560-b248-8d0c1b6e94e2") : object "kube-system"/"original-pull-secret" not registered Sep 08 10:17:05.755111 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:05.755031 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs podName:db4c4c5b-19b6-439a-bcd2-0631f7626479 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:06.755025072 +0000 UTC m=+3.124290348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs") pod "network-metrics-daemon-pr8g9" (UID: "db4c4c5b-19b6-439a-bcd2-0631f7626479") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:17:05.987193 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:05.987105 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 10:17:06.100764 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.100678 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 10:12:05 +0000 UTC" deadline="2028-04-05 03:50:55.544243133 +0000 UTC" Sep 08 10:17:06.100764 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.100757 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13793h33m49.443491956s" Sep 08 10:17:06.140327 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.140192 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:06.140527 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:06.140348 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:06.145954 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.145895 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-203.ec2.internal" event={"ID":"f4f934834e3ed193d36ec5ddfb5f38fb","Type":"ContainerStarted","Data":"a465d55ba4ff7ea592cda6f2b25b9f05e7ed39777af48687b083dc3c926bdde5"} Sep 08 10:17:06.148336 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.148305 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal" event={"ID":"5ff941279daf8a3e1f7d0667779a0142","Type":"ContainerStarted","Data":"f2d63f2fdde295f4931db473fbfbed14595b354a4a885204c5f245220b826381"} Sep 08 10:17:06.150004 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.149977 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" event={"ID":"08fd25ee-4cf2-467f-aa66-cb3f454c528c","Type":"ContainerStarted","Data":"a3b867b107e13daa364bbce3e3bd8b6cdeb0e99dff78fcfd42e5fbcd3363900a"} Sep 08 10:17:06.152730 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.152440 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-552hj" event={"ID":"6ee8913a-efe5-42f7-bf88-3a0439a828ba","Type":"ContainerStarted","Data":"eae40cab6d2c3cfa46c38c8054c80d3e5b7d1e471d3dd89ed9da0c297490cf17"} Sep 08 10:17:06.153299 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:06.153263 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"tuned\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" podUID="08fd25ee-4cf2-467f-aa66-cb3f454c528c" Sep 08 10:17:06.154509 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.154479 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" event={"ID":"b72101be-602c-4ea1-85fc-9baade17727b","Type":"ContainerStarted","Data":"e79953798864ab85522344798919f34b2914183dd43e10eecc1fd021c86a6f24"} Sep 08 10:17:06.159501 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.159469 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wkwnr" event={"ID":"4425cf1f-0af5-4990-9169-4ce1848e739d","Type":"ContainerStarted","Data":"aaab16d59cebd334304d7f9a285636b8c88f269adc27ceb66d6ea578879dbf10"} Sep 08 10:17:06.163322 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.163292 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qhxw7" event={"ID":"63e77b90-9f35-4b51-a977-290598c60b83","Type":"ContainerStarted","Data":"1f91f2fe2ad2b871540f8441ed6adbf5de13ab77911389af1d203f986ccaaf4a"} Sep 08 10:17:06.165462 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.165415 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4mxks" event={"ID":"a910ad03-91d3-448e-bf66-56effa2459f3","Type":"ContainerStarted","Data":"0b6566885bfd7bd1c74b1185b3ef12c3d89e7b134f5e89bbfd12ba2e21146063"} Sep 08 10:17:06.167389 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.167301 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pv25g" event={"ID":"919f3306-bc79-43be-92f0-2b46f301d6f7","Type":"ContainerStarted","Data":"cd8278a69d743a632354d3ce9e5876fe2d25cde039746022803832168fc3ea06"} Sep 08 10:17:06.169957 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.169930 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lw9vz" event={"ID":"e9f96208-ea87-4822-ae72-9c07e5d5fa8f","Type":"ContainerStarted","Data":"d5aaea3d7fe4eb18e773b95e52d11893b2b21a6ffd1c7728b098537a08f89a1b"} Sep 08 10:17:06.173750 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.173723 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" event={"ID":"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217","Type":"ContainerStarted","Data":"c5ff6b9c995a4e5cbbe4dec54e4738e9332aa58b9e31553c7e1efae744ade81b"} Sep 08 10:17:06.662460 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.662415 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqwvr\" (UniqueName: \"kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr\") pod \"network-check-target-xlksv\" (UID: \"0d5943f3-ba9e-4dc9-91c1-9787f83d5305\") " pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:06.662691 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:06.662664 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 10:17:06.662691 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:06.662687 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 10:17:06.662804 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:06.662701 2561 projected.go:196] Error preparing data for projected volume kube-api-access-hqwvr for pod openshift-network-diagnostics/network-check-target-xlksv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:17:06.662804 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:06.662775 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr podName:0d5943f3-ba9e-4dc9-91c1-9787f83d5305 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:08.662754021 +0000 UTC m=+5.032019299 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hqwvr" (UniqueName: "kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr") pod "network-check-target-xlksv" (UID: "0d5943f3-ba9e-4dc9-91c1-9787f83d5305") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:17:06.763965 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.763102 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret\") pod \"global-pull-secret-syncer-56snn\" (UID: \"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2\") " pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:06.763965 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:06.763179 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs\") pod \"network-metrics-daemon-pr8g9\" (UID: \"db4c4c5b-19b6-439a-bcd2-0631f7626479\") " pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:06.763965 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:06.763357 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:17:06.763965 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:06.763430 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs podName:db4c4c5b-19b6-439a-bcd2-0631f7626479 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:08.763408991 +0000 UTC m=+5.132674293 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs") pod "network-metrics-daemon-pr8g9" (UID: "db4c4c5b-19b6-439a-bcd2-0631f7626479") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:17:06.763965 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:06.763882 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 10:17:06.763965 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:06.763938 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret podName:e0ef2303-d3cc-4560-b248-8d0c1b6e94e2 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:08.763921705 +0000 UTC m=+5.133186999 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret") pod "global-pull-secret-syncer-56snn" (UID: "e0ef2303-d3cc-4560-b248-8d0c1b6e94e2") : object "kube-system"/"original-pull-secret" not registered Sep 08 10:17:07.090437 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:07.090354 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 10:17:07.101777 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:07.101689 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 10:12:05 +0000 UTC" deadline="2028-06-18 07:43:08.273495498 +0000 UTC" Sep 08 10:17:07.101777 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:07.101747 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15573h26m1.171752507s" Sep 08 10:17:07.140119 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:07.140082 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:07.140119 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:07.140127 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:07.140379 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:07.140231 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:07.140438 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:07.140388 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:07.508804 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:07.508724 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 10:17:08.139926 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:08.139875 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:08.140384 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:08.140030 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:08.680141 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:08.680097 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqwvr\" (UniqueName: \"kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr\") pod \"network-check-target-xlksv\" (UID: \"0d5943f3-ba9e-4dc9-91c1-9787f83d5305\") " pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:08.680340 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:08.680281 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 10:17:08.680340 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:08.680311 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 10:17:08.680340 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:08.680325 2561 projected.go:196] Error preparing data for projected volume kube-api-access-hqwvr for pod openshift-network-diagnostics/network-check-target-xlksv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:17:08.680531 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:08.680398 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr podName:0d5943f3-ba9e-4dc9-91c1-9787f83d5305 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:12.680377418 +0000 UTC m=+9.049642713 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hqwvr" (UniqueName: "kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr") pod "network-check-target-xlksv" (UID: "0d5943f3-ba9e-4dc9-91c1-9787f83d5305") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:17:08.780936 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:08.780888 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret\") pod \"global-pull-secret-syncer-56snn\" (UID: \"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2\") " pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:08.781138 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:08.780963 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs\") pod \"network-metrics-daemon-pr8g9\" (UID: \"db4c4c5b-19b6-439a-bcd2-0631f7626479\") " pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:08.781138 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:08.781070 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 10:17:08.781271 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:08.781157 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret podName:e0ef2303-d3cc-4560-b248-8d0c1b6e94e2 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:12.781140097 +0000 UTC m=+9.150405378 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret") pod "global-pull-secret-syncer-56snn" (UID: "e0ef2303-d3cc-4560-b248-8d0c1b6e94e2") : object "kube-system"/"original-pull-secret" not registered Sep 08 10:17:08.781271 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:08.781076 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:17:08.781271 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:08.781266 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs podName:db4c4c5b-19b6-439a-bcd2-0631f7626479 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:12.781244779 +0000 UTC m=+9.150510055 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs") pod "network-metrics-daemon-pr8g9" (UID: "db4c4c5b-19b6-439a-bcd2-0631f7626479") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:17:09.139989 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:09.139949 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:09.139989 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:09.139979 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:09.140490 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:09.140091 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:09.140490 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:09.140237 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:10.142426 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:10.142387 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:10.143066 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:10.142548 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:10.185569 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:10.185491 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qhxw7" event={"ID":"63e77b90-9f35-4b51-a977-290598c60b83","Type":"ContainerStarted","Data":"ec26e33c01bbbadd11a3b71495907c05230fdc04239bbdfc1f4298a7a28d7bf6"} Sep 08 10:17:10.191062 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:10.189758 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lw9vz" event={"ID":"e9f96208-ea87-4822-ae72-9c07e5d5fa8f","Type":"ContainerStarted","Data":"5b63c326382ef225baab7d6fc54650e5f02c0f4ea5a0c741e645910888d5f888"} Sep 08 10:17:10.196067 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:10.195474 2561 generic.go:356] "Generic (PLEG): container finished" podID="6ee8913a-efe5-42f7-bf88-3a0439a828ba" containerID="5a27fe061dc044e8ceb57c999e5c3a1a421d5eca55cef5f988e35642aa5fe1d5" exitCode=0 Sep 08 10:17:10.196067 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:10.195571 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-552hj" event={"ID":"6ee8913a-efe5-42f7-bf88-3a0439a828ba","Type":"ContainerDied","Data":"5a27fe061dc044e8ceb57c999e5c3a1a421d5eca55cef5f988e35642aa5fe1d5"} Sep 08 10:17:10.202412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:10.202332 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" event={"ID":"b72101be-602c-4ea1-85fc-9baade17727b","Type":"ContainerStarted","Data":"5490d6faed127b355974c47164c688621dcd3a21e9705a9b2f47083599244ff3"} Sep 08 10:17:10.231322 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:10.230740 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qhxw7" podStartSLOduration=1.140001215 podStartE2EDuration="5.230715236s" totalImagesPullingTime="4.090714021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:17:10.211571263 +0000 UTC m=+6.580836558" watchObservedRunningTime="2026-09-08 10:17:10.230715236 +0000 UTC m=+6.599980532" Sep 08 10:17:10.231525 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:10.231471 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lw9vz" podStartSLOduration=1.0830636 podStartE2EDuration="5.231438765s" totalImagesPullingTime="4.148375165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:17:10.230823984 +0000 UTC m=+6.600089286" watchObservedRunningTime="2026-09-08 10:17:10.231438765 +0000 UTC m=+6.600704068" Sep 08 10:17:10.853657 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:10.853620 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qhxw7" Sep 08 10:17:10.854156 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:10.854133 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qhxw7" Sep 08 10:17:11.140310 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:11.140219 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:11.140310 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:11.140282 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:11.140550 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:11.140387 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:11.140550 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:11.140515 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:12.139451 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:12.139413 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:12.139929 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:12.139597 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:12.208012 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:12.207981 2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 08 10:17:12.208187 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:12.207976 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wkwnr" event={"ID":"4425cf1f-0af5-4990-9169-4ce1848e739d","Type":"ContainerStarted","Data":"5266a49f00152792d0e42ba93e64282ad878d0ee41f2e3edf124e5d7bbbe5f7f"} Sep 08 10:17:12.227917 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:12.227854 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wkwnr" podStartSLOduration=3.056128992 podStartE2EDuration="7.227831252s" totalImagesPullingTime="4.17170226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:17:12.227134865 +0000 UTC m=+8.596400166" watchObservedRunningTime="2026-09-08 10:17:12.227831252 +0000 UTC m=+8.597096551" Sep 08 10:17:12.716154 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:12.716113 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqwvr\" (UniqueName: \"kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr\") pod \"network-check-target-xlksv\" (UID: \"0d5943f3-ba9e-4dc9-91c1-9787f83d5305\") " pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:12.716340 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:12.716251 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 10:17:12.716340 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:12.716279 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 10:17:12.716340 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:12.716294 2561 projected.go:196] Error preparing data for projected volume kube-api-access-hqwvr for pod openshift-network-diagnostics/network-check-target-xlksv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:17:12.716487 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:12.716363 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr podName:0d5943f3-ba9e-4dc9-91c1-9787f83d5305 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:20.716342178 +0000 UTC m=+17.085607468 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hqwvr" (UniqueName: "kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr") pod "network-check-target-xlksv" (UID: "0d5943f3-ba9e-4dc9-91c1-9787f83d5305") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:17:12.816557 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:12.816519 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret\") pod \"global-pull-secret-syncer-56snn\" (UID: \"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2\") " pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:12.816742 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:12.816601 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs\") pod \"network-metrics-daemon-pr8g9\" (UID: \"db4c4c5b-19b6-439a-bcd2-0631f7626479\") " pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:12.816808 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:12.816744 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:17:12.816808 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:12.816806 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs podName:db4c4c5b-19b6-439a-bcd2-0631f7626479 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:20.816788255 +0000 UTC m=+17.186053538 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs") pod "network-metrics-daemon-pr8g9" (UID: "db4c4c5b-19b6-439a-bcd2-0631f7626479") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:17:12.817125 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:12.817025 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 10:17:12.817125 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:12.817093 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret podName:e0ef2303-d3cc-4560-b248-8d0c1b6e94e2 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:20.817073687 +0000 UTC m=+17.186338974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret") pod "global-pull-secret-syncer-56snn" (UID: "e0ef2303-d3cc-4560-b248-8d0c1b6e94e2") : object "kube-system"/"original-pull-secret" not registered Sep 08 10:17:13.080692 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:13.080662 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 08 10:17:13.103082 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:13.102804 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T10:17:13.080688083Z","UUID":"5f8a57a7-60f6-42d5-ae65-8e73ceb961bc","Handler":null,"Name":"","Endpoint":""} Sep 08 10:17:13.104647 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:13.104626 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 08 10:17:13.104766 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:13.104664 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 08 10:17:13.139787 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:13.139756 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:13.139787 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:13.139782 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:13.140363 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:13.139895 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:13.140363 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:13.140047 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:13.212687 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:13.212510 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" event={"ID":"b72101be-602c-4ea1-85fc-9baade17727b","Type":"ContainerStarted","Data":"24cfb7f244ef21042143460bbb7b7e2cd2d4d8cb6d3e74beb31024accc4e9d11"} Sep 08 10:17:13.214823 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:13.214784 2561 generic.go:356] "Generic (PLEG): container finished" podID="5ff941279daf8a3e1f7d0667779a0142" containerID="f227f7f2d65ec1f77f0881de12cb3fb44774f21e7550acadfba129ae9b100e57" exitCode=0 Sep 08 10:17:13.215333 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:13.215304 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal" event={"ID":"5ff941279daf8a3e1f7d0667779a0142","Type":"ContainerDied","Data":"f227f7f2d65ec1f77f0881de12cb3fb44774f21e7550acadfba129ae9b100e57"} Sep 08 10:17:14.141915 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:14.141192 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:14.141915 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:14.141333 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:14.287610 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:14.287566 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qhxw7" Sep 08 10:17:14.287742 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:14.287710 2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 08 10:17:14.289253 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:14.289231 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qhxw7" Sep 08 10:17:15.140415 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:15.140160 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:15.140630 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:15.140213 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:15.140630 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:15.140525 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:15.140726 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:15.140645 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:15.221841 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:15.221802 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-203.ec2.internal" event={"ID":"f4f934834e3ed193d36ec5ddfb5f38fb","Type":"ContainerStarted","Data":"4e41c0d507e38bd18721cf4be5a354e97346115b1c452c660b51a2f2c8bae720"} Sep 08 10:17:15.225393 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:15.225320 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal" event={"ID":"5ff941279daf8a3e1f7d0667779a0142","Type":"ContainerStarted","Data":"3944149682b3710c8e78d0f6b62eb64f7dc369ba8394a314a27de18b5493a5d0"} Sep 08 10:17:15.239975 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:15.239931 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-203.ec2.internal" podStartSLOduration=10.239913063 podStartE2EDuration="10.239913063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:17:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:17:15.239630915 +0000 UTC m=+11.608896214" watchObservedRunningTime="2026-09-08 10:17:15.239913063 +0000 UTC m=+11.609178360" Sep 08 10:17:16.140314 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:16.140280 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:16.140553 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:16.140435 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:17.139787 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:17.139751 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:17.140303 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:17.139882 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:17.140303 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:17.139912 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:17.140303 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:17.140023 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:18.140381 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:18.140343 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:18.140859 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:18.140806 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:18.165172 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:18.165124 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-203.ec2.internal" podStartSLOduration=13.16511002 podStartE2EDuration="13.16511002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:17:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:17:15.256876451 +0000 UTC m=+11.626141756" watchObservedRunningTime="2026-09-08 10:17:18.16511002 +0000 UTC m=+14.534375316" Sep 08 10:17:19.139298 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:19.139266 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:19.139298 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:19.139298 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:19.139517 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:19.139393 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:19.139602 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:19.139538 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:20.139621 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:20.139583 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:20.140065 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:20.139739 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:20.782180 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:20.782145 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqwvr\" (UniqueName: \"kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr\") pod \"network-check-target-xlksv\" (UID: \"0d5943f3-ba9e-4dc9-91c1-9787f83d5305\") " pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:20.782347 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:20.782310 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 10:17:20.782347 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:20.782337 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 10:17:20.782427 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:20.782353 2561 projected.go:196] Error preparing data for projected volume kube-api-access-hqwvr for pod openshift-network-diagnostics/network-check-target-xlksv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:17:20.782427 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:20.782407 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr podName:0d5943f3-ba9e-4dc9-91c1-9787f83d5305 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:36.782389782 +0000 UTC m=+33.151655058 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hqwvr" (UniqueName: "kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr") pod "network-check-target-xlksv" (UID: "0d5943f3-ba9e-4dc9-91c1-9787f83d5305") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:17:20.883323 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:20.883285 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret\") pod \"global-pull-secret-syncer-56snn\" (UID: \"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2\") " pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:20.883484 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:20.883351 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs\") pod \"network-metrics-daemon-pr8g9\" (UID: \"db4c4c5b-19b6-439a-bcd2-0631f7626479\") " pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:20.883484 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:20.883456 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 10:17:20.883612 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:20.883543 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret podName:e0ef2303-d3cc-4560-b248-8d0c1b6e94e2 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:36.88350677 +0000 UTC m=+33.252772063 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret") pod "global-pull-secret-syncer-56snn" (UID: "e0ef2303-d3cc-4560-b248-8d0c1b6e94e2") : object "kube-system"/"original-pull-secret" not registered Sep 08 10:17:20.883612 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:20.883462 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:17:20.883612 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:20.883606 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs podName:db4c4c5b-19b6-439a-bcd2-0631f7626479 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:36.883588077 +0000 UTC m=+33.252853355 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs") pod "network-metrics-daemon-pr8g9" (UID: "db4c4c5b-19b6-439a-bcd2-0631f7626479") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:17:21.140235 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:21.140145 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:21.140701 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:21.140145 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:21.140701 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:21.140277 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:21.140701 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:21.140372 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:22.140366 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:22.140325 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:22.140786 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:22.140473 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:23.139470 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:23.139429 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:23.139669 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:23.139428 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:23.139669 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:23.139580 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:23.139669 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:23.139654 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:24.140285 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:24.140251 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:24.140736 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:24.140350 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:25.140092 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:25.140054 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:25.140279 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:25.140054 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:25.140279 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:25.140169 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:25.140279 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:25.140251 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:26.140222 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:26.140187 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:26.140376 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:26.140306 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:27.140082 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:27.140042 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:27.140082 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:27.140072 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:27.140377 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:27.140163 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:27.140377 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:27.140274 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:28.142929 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:28.142900 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:28.143411 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:28.143002 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:29.139690 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.139667 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:29.139773 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:29.139758 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:29.139827 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.139811 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:29.139906 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:29.139892 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:29.258872 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.258840 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" event={"ID":"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217","Type":"ContainerStarted","Data":"7593597cd43557146795e83c6fe5edc859b8b06c0ec05b23b1642f4e24e7d0da"} Sep 08 10:17:29.259387 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.258960 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" event={"ID":"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217","Type":"ContainerStarted","Data":"c3962a1c1bcc1e963ce9b68c1c73f25b9870d0d6f702b99411a0ccdb035e7bde"} Sep 08 10:17:29.259387 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.259005 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" event={"ID":"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217","Type":"ContainerStarted","Data":"901bf6225113b62f30b2a1ce97097985476f67d766f489aec63a22c592e7f7a2"} Sep 08 10:17:29.259387 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.259052 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" event={"ID":"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217","Type":"ContainerStarted","Data":"d341abfd05e87bcf8e55ac398e8db85355ed9d0f5e7cbb83cf98887a29cc7f24"} Sep 08 10:17:29.259387 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.259096 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" event={"ID":"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217","Type":"ContainerStarted","Data":"03acc9b534533e28a5adcac0d57b742c37b3eb2e23005a9a3dc0cf8000bf5d68"} Sep 08 10:17:29.259387 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.259143 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" event={"ID":"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217","Type":"ContainerStarted","Data":"e1cd0531d1aad56d41c8142f3ae9b92fdf91d794838edcd54e0622422f56851f"} Sep 08 10:17:29.260370 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.260346 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" event={"ID":"08fd25ee-4cf2-467f-aa66-cb3f454c528c","Type":"ContainerStarted","Data":"8fddcdfbff7bcc9b4f4893b194d9f12b9fd8a2962a5235e4cfaf9f3152efdc66"} Sep 08 10:17:29.262315 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.262290 2561 generic.go:356] "Generic (PLEG): container finished" podID="6ee8913a-efe5-42f7-bf88-3a0439a828ba" containerID="dedd3c3d70afde42e6ba202d131d292c3ca295e5ffad212ce5ca75ae2c5f9d5c" exitCode=0 Sep 08 10:17:29.262429 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.262369 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-552hj" event={"ID":"6ee8913a-efe5-42f7-bf88-3a0439a828ba","Type":"ContainerDied","Data":"dedd3c3d70afde42e6ba202d131d292c3ca295e5ffad212ce5ca75ae2c5f9d5c"} Sep 08 10:17:29.264671 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.264644 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" event={"ID":"b72101be-602c-4ea1-85fc-9baade17727b","Type":"ContainerStarted","Data":"37aa7cc59dc76442327d5a9e67cb4736203535f9d4435d8b8c4a3e07968e6ae3"} Sep 08 10:17:29.266419 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.266211 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4mxks" event={"ID":"a910ad03-91d3-448e-bf66-56effa2459f3","Type":"ContainerStarted","Data":"cc40912b5f2e11f17dfcbefb0d6d9f1c5216123825209c4c550c8aec00b450ef"} Sep 08 10:17:29.267641 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.267622 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pv25g" event={"ID":"919f3306-bc79-43be-92f0-2b46f301d6f7","Type":"ContainerStarted","Data":"8e600895bfd71c97f07389579e767cf1b1e308fad286e1024ad6da42fc399641"} Sep 08 10:17:29.280801 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.280731 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lsnsb" podStartSLOduration=0.963242043 podStartE2EDuration="24.28072001s" totalImagesPullingTime="23.317477967s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-09-08 10:17:29.280382665 +0000 UTC m=+25.649647962" watchObservedRunningTime="2026-09-08 10:17:29.28072001 +0000 UTC m=+25.649985307" Sep 08 10:17:29.302423 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.302377 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pjw8" podStartSLOduration=1.582775903 podStartE2EDuration="24.302359645s" totalImagesPullingTime="22.719583742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:17:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:17:29.301509922 +0000 UTC m=+25.670775218" watchObservedRunningTime="2026-09-08 10:17:29.302359645 +0000 UTC m=+25.671624943" Sep 08 10:17:29.356957 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:29.356907 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4mxks" podStartSLOduration=1.032642544 podStartE2EDuration="24.356894192s" totalImagesPullingTime="23.324251648s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:17:29.328520725 +0000 UTC m=+25.697786022" watchObservedRunningTime="2026-09-08 10:17:29.356894192 +0000 UTC m=+25.726159489" Sep 08 10:17:30.139577 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:30.139502 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:30.139721 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:30.139625 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:31.139484 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:31.139457 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:31.139854 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:31.139457 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:31.139854 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:31.139555 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:31.139854 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:31.139613 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:31.273959 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:31.273938 2561 generic.go:356] "Generic (PLEG): container finished" podID="6ee8913a-efe5-42f7-bf88-3a0439a828ba" containerID="62981bc733ec36b8372d96523a83b1bd3e96ac000a6e844e6d57fead20a8e33c" exitCode=0 Sep 08 10:17:31.274056 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:31.273984 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-552hj" event={"ID":"6ee8913a-efe5-42f7-bf88-3a0439a828ba","Type":"ContainerDied","Data":"62981bc733ec36b8372d96523a83b1bd3e96ac000a6e844e6d57fead20a8e33c"} Sep 08 10:17:31.296703 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:31.296660 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pv25g" podStartSLOduration=2.960722902 podStartE2EDuration="26.296647682s" totalImagesPullingTime="23.33592478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:17:29.386233418 +0000 UTC m=+25.755498714" watchObservedRunningTime="2026-09-08 10:17:31.296647682 +0000 UTC m=+27.665913008" Sep 08 10:17:32.142849 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:32.142825 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:32.143296 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:32.142954 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:32.278776 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:32.278749 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" event={"ID":"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217","Type":"ContainerStarted","Data":"ec9aecee90726f6cc72af46c8aa52718393effcacb983d791720ede624d90862"} Sep 08 10:17:32.280939 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:32.280916 2561 generic.go:356] "Generic (PLEG): container finished" podID="6ee8913a-efe5-42f7-bf88-3a0439a828ba" containerID="148abd241211da1f096e22a0d60dd99b6aa6b92af7e75b256e609f90f4f007c5" exitCode=0 Sep 08 10:17:32.281036 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:32.280951 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-552hj" event={"ID":"6ee8913a-efe5-42f7-bf88-3a0439a828ba","Type":"ContainerDied","Data":"148abd241211da1f096e22a0d60dd99b6aa6b92af7e75b256e609f90f4f007c5"} Sep 08 10:17:33.139275 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:33.139251 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:33.139418 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:33.139250 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:33.139418 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:33.139408 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:33.139500 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:33.139336 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:34.140657 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:34.140424 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:34.141192 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:34.140759 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:34.287259 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:34.287230 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" event={"ID":"d7f1ba8a-c265-4a99-ae50-00c5b2b3c217","Type":"ContainerStarted","Data":"34fd80e5272cb463a19c49c113837168c793393ef9713c837a6d72d64ccb365e"} Sep 08 10:17:34.287505 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:34.287478 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:34.287664 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:34.287558 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:34.303466 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:34.303436 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:34.328454 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:34.328413 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" podStartSLOduration=5.954701706 podStartE2EDuration="29.328402305s" totalImagesPullingTime="23.373700599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:17:34.328297422 +0000 UTC m=+30.697562719" watchObservedRunningTime="2026-09-08 10:17:34.328402305 +0000 UTC m=+30.697667615" Sep 08 10:17:35.140249 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:35.140222 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:35.140399 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:35.140222 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:35.140399 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:35.140314 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:35.140399 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:35.140380 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:35.271739 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:35.271712 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-56snn"] Sep 08 10:17:35.279705 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:35.279683 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pr8g9"] Sep 08 10:17:35.279801 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:35.279760 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:35.279860 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:35.279839 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:35.282455 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:35.282435 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xlksv"] Sep 08 10:17:35.289698 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:35.289677 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:35.289776 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:35.289757 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:35.289836 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:35.289684 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:35.290202 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:35.290183 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:35.290342 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:35.290170 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:35.306545 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:35.306506 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:17:36.793169 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:36.793132 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqwvr\" (UniqueName: \"kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr\") pod \"network-check-target-xlksv\" (UID: \"0d5943f3-ba9e-4dc9-91c1-9787f83d5305\") " pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:36.793670 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:36.793286 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 10:17:36.793670 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:36.793306 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 10:17:36.793670 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:36.793316 2561 projected.go:196] Error preparing data for projected volume kube-api-access-hqwvr for pod openshift-network-diagnostics/network-check-target-xlksv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:17:36.793670 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:36.793364 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr podName:0d5943f3-ba9e-4dc9-91c1-9787f83d5305 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:08.793350354 +0000 UTC m=+65.162615643 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hqwvr" (UniqueName: "kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr") pod "network-check-target-xlksv" (UID: "0d5943f3-ba9e-4dc9-91c1-9787f83d5305") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 10:17:36.894301 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:36.894257 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret\") pod \"global-pull-secret-syncer-56snn\" (UID: \"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2\") " pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:36.894301 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:36.894312 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs\") pod \"network-metrics-daemon-pr8g9\" (UID: \"db4c4c5b-19b6-439a-bcd2-0631f7626479\") " pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:36.894514 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:36.894407 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:17:36.894514 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:36.894409 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 10:17:36.894514 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:36.894461 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs podName:db4c4c5b-19b6-439a-bcd2-0631f7626479 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:08.894447089 +0000 UTC m=+65.263712365 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs") pod "network-metrics-daemon-pr8g9" (UID: "db4c4c5b-19b6-439a-bcd2-0631f7626479") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 10:17:36.894514 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:36.894475 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret podName:e0ef2303-d3cc-4560-b248-8d0c1b6e94e2 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:08.894468809 +0000 UTC m=+65.263734085 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret") pod "global-pull-secret-syncer-56snn" (UID: "e0ef2303-d3cc-4560-b248-8d0c1b6e94e2") : object "kube-system"/"original-pull-secret" not registered Sep 08 10:17:37.140212 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:37.140118 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:37.140212 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:37.140141 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:37.140560 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:37.140289 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:37.140560 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:37.140332 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:37.140560 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:37.140410 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:37.140560 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:37.140449 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:39.139361 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:39.139328 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:39.139800 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:39.139417 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:39.139800 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:39.139421 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:39.139800 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:39.139458 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:39.139800 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:39.139561 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:39.139800 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:39.139633 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:41.139975 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.139935 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:41.140439 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.139987 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:41.140439 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.140049 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:41.140439 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:41.140067 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-56snn" podUID="e0ef2303-d3cc-4560-b248-8d0c1b6e94e2" Sep 08 10:17:41.140439 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:41.140147 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xlksv" podUID="0d5943f3-ba9e-4dc9-91c1-9787f83d5305" Sep 08 10:17:41.140439 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:41.140253 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8g9" podUID="db4c4c5b-19b6-439a-bcd2-0631f7626479" Sep 08 10:17:41.474609 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.474578 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-203.ec2.internal" event="NodeReady" Sep 08 10:17:41.474782 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.474747 2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 08 10:17:41.523522 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.523493 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59557ddf67-5k75j"] Sep 08 10:17:41.543628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.543563 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8nhg6"] Sep 08 10:17:41.543628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.543586 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.548002 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.547787 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 08 10:17:41.548002 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.547908 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 08 10:17:41.548203 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.548122 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 08 10:17:41.548262 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.548214 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-svksl\"" Sep 08 10:17:41.554657 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.554627 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 08 10:17:41.571185 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.571154 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59557ddf67-5k75j"] Sep 08 10:17:41.571344 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.571195 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8dfxz"] Sep 08 10:17:41.571344 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.571264 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:41.574317 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.574273 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f7bmn\"" Sep 08 10:17:41.574521 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.574498 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 08 10:17:41.574605 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.574566 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 08 10:17:41.589323 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.589296 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8dfxz"] Sep 08 10:17:41.589454 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.589338 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8nhg6"] Sep 08 10:17:41.589454 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.589405 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8dfxz" Sep 08 10:17:41.592159 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.592130 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9klnr\"" Sep 08 10:17:41.592159 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.592146 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 08 10:17:41.592299 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.592145 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 08 10:17:41.592299 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.592240 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 08 10:17:41.625943 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.625899 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2dd92822-b06b-4eaa-8cf7-cc210c5de102-installation-pull-secrets\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.626121 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.625949 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-certificates\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.626121 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.626069 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.626121 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.626105 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-bound-sa-token\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.626281 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.626134 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prr8d\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-kube-api-access-prr8d\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.626281 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.626159 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2dd92822-b06b-4eaa-8cf7-cc210c5de102-trusted-ca\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.626281 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.626191 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2dd92822-b06b-4eaa-8cf7-cc210c5de102-image-registry-private-configuration\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.626281 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.626223 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2dd92822-b06b-4eaa-8cf7-cc210c5de102-ca-trust-extracted\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.726933 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.726841 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2dd92822-b06b-4eaa-8cf7-cc210c5de102-trusted-ca\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.726933 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.726887 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:41.726933 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.726917 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2dd92822-b06b-4eaa-8cf7-cc210c5de102-image-registry-private-configuration\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.727158 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.727110 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d3689cda-0bc7-4455-8530-c685d3a0fe94-config-volume\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:41.727158 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.727148 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3689cda-0bc7-4455-8530-c685d3a0fe94-tmp-dir\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:41.727260 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.727188 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q87dp\" (UniqueName: \"kubernetes.io/projected/3c246116-fdc5-4fde-8f72-8835d9718f7d-kube-api-access-q87dp\") pod \"ingress-canary-8dfxz\" (UID: \"3c246116-fdc5-4fde-8f72-8835d9718f7d\") " pod="openshift-ingress-canary/ingress-canary-8dfxz" Sep 08 10:17:41.727260 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.727222 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2dd92822-b06b-4eaa-8cf7-cc210c5de102-ca-trust-extracted\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.727357 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.727258 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96nxj\" (UniqueName: \"kubernetes.io/projected/d3689cda-0bc7-4455-8530-c685d3a0fe94-kube-api-access-96nxj\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:41.727357 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.727302 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2dd92822-b06b-4eaa-8cf7-cc210c5de102-installation-pull-secrets\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.727357 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.727330 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-certificates\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.727357 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.727355 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c246116-fdc5-4fde-8f72-8835d9718f7d-cert\") pod \"ingress-canary-8dfxz\" (UID: \"3c246116-fdc5-4fde-8f72-8835d9718f7d\") " pod="openshift-ingress-canary/ingress-canary-8dfxz" Sep 08 10:17:41.727596 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.727442 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.727596 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.727474 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-bound-sa-token\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.727596 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.727499 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prr8d\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-kube-api-access-prr8d\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.727742 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.727594 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2dd92822-b06b-4eaa-8cf7-cc210c5de102-ca-trust-extracted\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.727742 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:41.727708 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 10:17:41.727742 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:41.727725 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59557ddf67-5k75j: secret "image-registry-tls" not found Sep 08 10:17:41.727877 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:41.727789 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls podName:2dd92822-b06b-4eaa-8cf7-cc210c5de102 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:42.227769123 +0000 UTC m=+38.597034413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls") pod "image-registry-59557ddf67-5k75j" (UID: "2dd92822-b06b-4eaa-8cf7-cc210c5de102") : secret "image-registry-tls" not found Sep 08 10:17:41.727982 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.727958 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-certificates\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.728078 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.728033 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2dd92822-b06b-4eaa-8cf7-cc210c5de102-trusted-ca\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.732126 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.732007 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2dd92822-b06b-4eaa-8cf7-cc210c5de102-installation-pull-secrets\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.732229 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.732018 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2dd92822-b06b-4eaa-8cf7-cc210c5de102-image-registry-private-configuration\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.737078 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.737048 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-bound-sa-token\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.737277 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.737261 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prr8d\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-kube-api-access-prr8d\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:41.828406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.828368 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q87dp\" (UniqueName: \"kubernetes.io/projected/3c246116-fdc5-4fde-8f72-8835d9718f7d-kube-api-access-q87dp\") pod \"ingress-canary-8dfxz\" (UID: \"3c246116-fdc5-4fde-8f72-8835d9718f7d\") " pod="openshift-ingress-canary/ingress-canary-8dfxz" Sep 08 10:17:41.828629 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.828499 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96nxj\" (UniqueName: \"kubernetes.io/projected/d3689cda-0bc7-4455-8530-c685d3a0fe94-kube-api-access-96nxj\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:41.828708 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.828626 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c246116-fdc5-4fde-8f72-8835d9718f7d-cert\") pod \"ingress-canary-8dfxz\" (UID: \"3c246116-fdc5-4fde-8f72-8835d9718f7d\") " pod="openshift-ingress-canary/ingress-canary-8dfxz" Sep 08 10:17:41.828777 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.828705 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:41.828777 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.828735 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d3689cda-0bc7-4455-8530-c685d3a0fe94-config-volume\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:41.828777 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.828758 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3689cda-0bc7-4455-8530-c685d3a0fe94-tmp-dir\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:41.828952 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:41.828876 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 10:17:41.828998 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:41.828954 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls podName:d3689cda-0bc7-4455-8530-c685d3a0fe94 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:42.328931986 +0000 UTC m=+38.698197278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls") pod "dns-default-8nhg6" (UID: "d3689cda-0bc7-4455-8530-c685d3a0fe94") : secret "dns-default-metrics-tls" not found Sep 08 10:17:41.829084 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.829057 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3689cda-0bc7-4455-8530-c685d3a0fe94-tmp-dir\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:41.829509 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.829466 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d3689cda-0bc7-4455-8530-c685d3a0fe94-config-volume\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:41.831591 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.831567 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3c246116-fdc5-4fde-8f72-8835d9718f7d-cert\") pod \"ingress-canary-8dfxz\" (UID: \"3c246116-fdc5-4fde-8f72-8835d9718f7d\") " pod="openshift-ingress-canary/ingress-canary-8dfxz" Sep 08 10:17:41.841992 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.841966 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96nxj\" (UniqueName: \"kubernetes.io/projected/d3689cda-0bc7-4455-8530-c685d3a0fe94-kube-api-access-96nxj\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:41.842157 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.842139 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q87dp\" (UniqueName: \"kubernetes.io/projected/3c246116-fdc5-4fde-8f72-8835d9718f7d-kube-api-access-q87dp\") pod \"ingress-canary-8dfxz\" (UID: \"3c246116-fdc5-4fde-8f72-8835d9718f7d\") " pod="openshift-ingress-canary/ingress-canary-8dfxz" Sep 08 10:17:41.899080 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:41.899041 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8dfxz" Sep 08 10:17:42.231286 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:42.231250 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:42.231989 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:42.231413 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 10:17:42.231989 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:42.231434 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59557ddf67-5k75j: secret "image-registry-tls" not found Sep 08 10:17:42.231989 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:42.231496 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls podName:2dd92822-b06b-4eaa-8cf7-cc210c5de102 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:43.231478248 +0000 UTC m=+39.600743529 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls") pod "image-registry-59557ddf67-5k75j" (UID: "2dd92822-b06b-4eaa-8cf7-cc210c5de102") : secret "image-registry-tls" not found Sep 08 10:17:42.231989 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:42.231910 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8dfxz"] Sep 08 10:17:42.235889 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:17:42.235865 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c246116_fdc5_4fde_8f72_8835d9718f7d.slice/crio-10d74d3e6089eea2158ca80179821f269b101adf062c72be1e1232bedd569eab WatchSource:0}: Error finding container 10d74d3e6089eea2158ca80179821f269b101adf062c72be1e1232bedd569eab: Status 404 returned error can't find the container with id 10d74d3e6089eea2158ca80179821f269b101adf062c72be1e1232bedd569eab Sep 08 10:17:42.307908 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:42.307827 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8dfxz" event={"ID":"3c246116-fdc5-4fde-8f72-8835d9718f7d","Type":"ContainerStarted","Data":"10d74d3e6089eea2158ca80179821f269b101adf062c72be1e1232bedd569eab"} Sep 08 10:17:42.332238 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:42.332208 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:42.332399 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:42.332345 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 10:17:42.332449 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:42.332414 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls podName:d3689cda-0bc7-4455-8530-c685d3a0fe94 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:43.332394591 +0000 UTC m=+39.701659866 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls") pod "dns-default-8nhg6" (UID: "d3689cda-0bc7-4455-8530-c685d3a0fe94") : secret "dns-default-metrics-tls" not found Sep 08 10:17:43.139710 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:43.139604 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:17:43.139710 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:43.139599 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:17:43.139909 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:43.139605 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:17:43.142509 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:43.142432 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 10:17:43.142509 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:43.142432 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 10:17:43.143213 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:43.142944 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2btxr\"" Sep 08 10:17:43.143213 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:43.143103 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 10:17:43.143213 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:43.143122 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ljzsh\"" Sep 08 10:17:43.143213 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:43.143176 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 10:17:43.240542 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:43.240492 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:43.241130 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:43.241050 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 10:17:43.241130 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:43.241067 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59557ddf67-5k75j: secret "image-registry-tls" not found Sep 08 10:17:43.241130 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:43.241131 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls podName:2dd92822-b06b-4eaa-8cf7-cc210c5de102 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:45.241112434 +0000 UTC m=+41.610377714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls") pod "image-registry-59557ddf67-5k75j" (UID: "2dd92822-b06b-4eaa-8cf7-cc210c5de102") : secret "image-registry-tls" not found Sep 08 10:17:43.341955 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:43.341923 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:43.342125 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:43.342111 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 10:17:43.342196 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:43.342172 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls podName:d3689cda-0bc7-4455-8530-c685d3a0fe94 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:45.342153786 +0000 UTC m=+41.711419064 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls") pod "dns-default-8nhg6" (UID: "d3689cda-0bc7-4455-8530-c685d3a0fe94") : secret "dns-default-metrics-tls" not found Sep 08 10:17:45.259468 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:45.259431 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:45.260009 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:45.259608 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 10:17:45.260009 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:45.259629 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59557ddf67-5k75j: secret "image-registry-tls" not found Sep 08 10:17:45.260009 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:45.259685 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls podName:2dd92822-b06b-4eaa-8cf7-cc210c5de102 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:49.259669614 +0000 UTC m=+45.628934903 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls") pod "image-registry-59557ddf67-5k75j" (UID: "2dd92822-b06b-4eaa-8cf7-cc210c5de102") : secret "image-registry-tls" not found Sep 08 10:17:45.360328 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:45.360291 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:45.360547 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:45.360418 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 10:17:45.360547 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:45.360496 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls podName:d3689cda-0bc7-4455-8530-c685d3a0fe94 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:49.360479905 +0000 UTC m=+45.729745181 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls") pod "dns-default-8nhg6" (UID: "d3689cda-0bc7-4455-8530-c685d3a0fe94") : secret "dns-default-metrics-tls" not found Sep 08 10:17:46.319142 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:46.319087 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8dfxz" event={"ID":"3c246116-fdc5-4fde-8f72-8835d9718f7d","Type":"ContainerStarted","Data":"bd9e7cf095d334c9f21db626d6a45b0f29710495ce8ed704c888caaa445c29bc"} Sep 08 10:17:46.322598 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:46.322565 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-552hj" event={"ID":"6ee8913a-efe5-42f7-bf88-3a0439a828ba","Type":"ContainerStarted","Data":"1d1eba457acd2f16ae57cddb0259ab36654d855e788772987e54bff65e983c19"} Sep 08 10:17:46.339353 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:46.339299 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8dfxz" podStartSLOduration=1.447218256 podStartE2EDuration="5.339281591s" totalImagesPullingTime="3.892063335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:17:46.338596977 +0000 UTC m=+42.707862275" watchObservedRunningTime="2026-09-08 10:17:46.339281591 +0000 UTC m=+42.708546891" Sep 08 10:17:47.327909 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:47.327877 2561 generic.go:356] "Generic (PLEG): container finished" podID="6ee8913a-efe5-42f7-bf88-3a0439a828ba" containerID="1d1eba457acd2f16ae57cddb0259ab36654d855e788772987e54bff65e983c19" exitCode=0 Sep 08 10:17:47.328336 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:47.327935 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-552hj" event={"ID":"6ee8913a-efe5-42f7-bf88-3a0439a828ba","Type":"ContainerDied","Data":"1d1eba457acd2f16ae57cddb0259ab36654d855e788772987e54bff65e983c19"} Sep 08 10:17:48.333033 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:48.333002 2561 generic.go:356] "Generic (PLEG): container finished" podID="6ee8913a-efe5-42f7-bf88-3a0439a828ba" containerID="a96d0d56f1230a926af59888bad039f72d0355e8fa4d5cf4f2ace5bc072534a5" exitCode=0 Sep 08 10:17:48.333396 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:48.333066 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-552hj" event={"ID":"6ee8913a-efe5-42f7-bf88-3a0439a828ba","Type":"ContainerDied","Data":"a96d0d56f1230a926af59888bad039f72d0355e8fa4d5cf4f2ace5bc072534a5"} Sep 08 10:17:49.290288 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:49.290241 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:49.290477 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:49.290424 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 10:17:49.290477 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:49.290449 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59557ddf67-5k75j: secret "image-registry-tls" not found Sep 08 10:17:49.290573 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:49.290506 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls podName:2dd92822-b06b-4eaa-8cf7-cc210c5de102 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:57.290489547 +0000 UTC m=+53.659754835 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls") pod "image-registry-59557ddf67-5k75j" (UID: "2dd92822-b06b-4eaa-8cf7-cc210c5de102") : secret "image-registry-tls" not found Sep 08 10:17:49.339035 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:49.338998 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-552hj" event={"ID":"6ee8913a-efe5-42f7-bf88-3a0439a828ba","Type":"ContainerStarted","Data":"3ed7de0bd8acf26c3dfc243f8c64d06b40d1743f6082c6e5dfa3141f5532d6f0"} Sep 08 10:17:49.390668 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:49.390624 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:49.390826 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:49.390782 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 10:17:49.390868 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:49.390846 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls podName:d3689cda-0bc7-4455-8530-c685d3a0fe94 nodeName:}" failed. No retries permitted until 2026-09-08 10:17:57.390831255 +0000 UTC m=+53.760096529 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls") pod "dns-default-8nhg6" (UID: "d3689cda-0bc7-4455-8530-c685d3a0fe94") : secret "dns-default-metrics-tls" not found Sep 08 10:17:57.349061 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:57.349013 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:17:57.349652 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:57.349183 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 10:17:57.349652 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:57.349206 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59557ddf67-5k75j: secret "image-registry-tls" not found Sep 08 10:17:57.349652 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:57.349284 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls podName:2dd92822-b06b-4eaa-8cf7-cc210c5de102 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:13.349262001 +0000 UTC m=+69.718527276 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls") pod "image-registry-59557ddf67-5k75j" (UID: "2dd92822-b06b-4eaa-8cf7-cc210c5de102") : secret "image-registry-tls" not found Sep 08 10:17:57.449592 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:17:57.449550 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:17:57.449768 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:57.449677 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 10:17:57.449768 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:17:57.449738 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls podName:d3689cda-0bc7-4455-8530-c685d3a0fe94 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:13.449724764 +0000 UTC m=+69.818990053 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls") pod "dns-default-8nhg6" (UID: "d3689cda-0bc7-4455-8530-c685d3a0fe94") : secret "dns-default-metrics-tls" not found Sep 08 10:18:07.305687 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:07.305658 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wx64r" Sep 08 10:18:07.345198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:07.345146 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-552hj" podStartSLOduration=24.01805596 podStartE2EDuration="1m2.345129624s" totalImagesPullingTime="38.327073664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:17:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:17:49.371222933 +0000 UTC m=+45.740488229" watchObservedRunningTime="2026-09-08 10:18:07.345129624 +0000 UTC m=+63.714394922" Sep 08 10:18:08.104762 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.104726 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-64bb84f4d-q4pg8"] Sep 08 10:18:08.106730 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.106705 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:08.115227 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.115205 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 08 10:18:08.115757 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.115735 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-lg649\"" Sep 08 10:18:08.115843 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.115783 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 08 10:18:08.115898 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.115733 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 08 10:18:08.115898 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.115735 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 08 10:18:08.116199 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.116182 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 08 10:18:08.116302 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.116186 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 08 10:18:08.137024 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.136990 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64bb84f4d-q4pg8"] Sep 08 10:18:08.225022 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.224987 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:08.225189 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.225030 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-stats-auth\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:08.225189 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.225050 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnvxs\" (UniqueName: \"kubernetes.io/projected/7484ddef-5576-43fe-857a-000cb91c4460-kube-api-access-vnvxs\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:08.225189 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.225086 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-default-certificate\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:08.225189 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.225131 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:08.325752 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.325716 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-default-certificate\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:08.325752 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.325756 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:08.326321 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.325825 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:08.326321 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.325845 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-stats-auth\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:08.326321 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.325861 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnvxs\" (UniqueName: \"kubernetes.io/projected/7484ddef-5576-43fe-857a-000cb91c4460-kube-api-access-vnvxs\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:08.326321 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:08.325969 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 10:18:08.326321 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:08.326020 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle podName:7484ddef-5576-43fe-857a-000cb91c4460 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:08.825999701 +0000 UTC m=+65.195264992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle") pod "router-default-64bb84f4d-q4pg8" (UID: "7484ddef-5576-43fe-857a-000cb91c4460") : configmap references non-existent config key: service-ca.crt Sep 08 10:18:08.326321 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:08.326053 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs podName:7484ddef-5576-43fe-857a-000cb91c4460 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:08.826030603 +0000 UTC m=+65.195295899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs") pod "router-default-64bb84f4d-q4pg8" (UID: "7484ddef-5576-43fe-857a-000cb91c4460") : secret "router-metrics-certs-default" not found Sep 08 10:18:08.329131 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.329102 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-default-certificate\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:08.329221 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.329131 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-stats-auth\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:08.337312 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.337290 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnvxs\" (UniqueName: \"kubernetes.io/projected/7484ddef-5576-43fe-857a-000cb91c4460-kube-api-access-vnvxs\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:08.830361 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.830320 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqwvr\" (UniqueName: \"kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr\") pod \"network-check-target-xlksv\" (UID: \"0d5943f3-ba9e-4dc9-91c1-9787f83d5305\") " pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:18:08.830567 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.830392 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:08.830567 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.830430 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:08.830644 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:08.830572 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 10:18:08.830644 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:08.830624 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs podName:7484ddef-5576-43fe-857a-000cb91c4460 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:09.830605328 +0000 UTC m=+66.199870605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs") pod "router-default-64bb84f4d-q4pg8" (UID: "7484ddef-5576-43fe-857a-000cb91c4460") : secret "router-metrics-certs-default" not found Sep 08 10:18:08.830722 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:08.830646 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle podName:7484ddef-5576-43fe-857a-000cb91c4460 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:09.830637874 +0000 UTC m=+66.199903148 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle") pod "router-default-64bb84f4d-q4pg8" (UID: "7484ddef-5576-43fe-857a-000cb91c4460") : configmap references non-existent config key: service-ca.crt Sep 08 10:18:08.833139 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.833116 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 10:18:08.843643 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.843621 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 10:18:08.854067 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.854040 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqwvr\" (UniqueName: \"kubernetes.io/projected/0d5943f3-ba9e-4dc9-91c1-9787f83d5305-kube-api-access-hqwvr\") pod \"network-check-target-xlksv\" (UID: \"0d5943f3-ba9e-4dc9-91c1-9787f83d5305\") " pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:18:08.931729 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.931704 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret\") pod \"global-pull-secret-syncer-56snn\" (UID: \"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2\") " pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:18:08.931840 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.931739 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs\") pod \"network-metrics-daemon-pr8g9\" (UID: \"db4c4c5b-19b6-439a-bcd2-0631f7626479\") " pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:18:08.936377 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.936355 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 10:18:08.936488 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.936366 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 10:18:08.942938 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:08.942919 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 08 10:18:08.943040 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:08.942974 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs podName:db4c4c5b-19b6-439a-bcd2-0631f7626479 nodeName:}" failed. No retries permitted until 2026-09-08 10:19:12.942957656 +0000 UTC m=+129.312222936 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs") pod "network-metrics-daemon-pr8g9" (UID: "db4c4c5b-19b6-439a-bcd2-0631f7626479") : secret "metrics-daemon-secret" not found Sep 08 10:18:08.945008 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.944986 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ef2303-d3cc-4560-b248-8d0c1b6e94e2-original-pull-secret\") pod \"global-pull-secret-syncer-56snn\" (UID: \"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2\") " pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:18:08.961181 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.961155 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-56snn" Sep 08 10:18:08.971672 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.971654 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2btxr\"" Sep 08 10:18:08.979963 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:08.979945 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:18:09.240971 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:09.240899 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-56snn"] Sep 08 10:18:09.243559 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:18:09.243517 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0ef2303_d3cc_4560_b248_8d0c1b6e94e2.slice/crio-9152b5f1dd47c69052bf20d140d9173dcd34c644d632ed19814fc4dd75b220a0 WatchSource:0}: Error finding container 9152b5f1dd47c69052bf20d140d9173dcd34c644d632ed19814fc4dd75b220a0: Status 404 returned error can't find the container with id 9152b5f1dd47c69052bf20d140d9173dcd34c644d632ed19814fc4dd75b220a0 Sep 08 10:18:09.266422 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:09.266398 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xlksv"] Sep 08 10:18:09.268864 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:18:09.268837 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d5943f3_ba9e_4dc9_91c1_9787f83d5305.slice/crio-dec9227c95a3fa85bf4daec85063d2a433851c62bb15362ba96bc272002f8011 WatchSource:0}: Error finding container dec9227c95a3fa85bf4daec85063d2a433851c62bb15362ba96bc272002f8011: Status 404 returned error can't find the container with id dec9227c95a3fa85bf4daec85063d2a433851c62bb15362ba96bc272002f8011 Sep 08 10:18:09.387300 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:09.387264 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xlksv" event={"ID":"0d5943f3-ba9e-4dc9-91c1-9787f83d5305","Type":"ContainerStarted","Data":"dec9227c95a3fa85bf4daec85063d2a433851c62bb15362ba96bc272002f8011"} Sep 08 10:18:09.388334 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:09.388314 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-56snn" event={"ID":"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2","Type":"ContainerStarted","Data":"9152b5f1dd47c69052bf20d140d9173dcd34c644d632ed19814fc4dd75b220a0"} Sep 08 10:18:09.838689 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:09.838656 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:09.838855 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:09.838701 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:09.838855 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:09.838815 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 10:18:09.838970 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:09.838854 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle podName:7484ddef-5576-43fe-857a-000cb91c4460 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:11.83883019 +0000 UTC m=+68.208095480 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle") pod "router-default-64bb84f4d-q4pg8" (UID: "7484ddef-5576-43fe-857a-000cb91c4460") : configmap references non-existent config key: service-ca.crt Sep 08 10:18:09.838970 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:09.838883 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs podName:7484ddef-5576-43fe-857a-000cb91c4460 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:11.838871307 +0000 UTC m=+68.208136585 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs") pod "router-default-64bb84f4d-q4pg8" (UID: "7484ddef-5576-43fe-857a-000cb91c4460") : secret "router-metrics-certs-default" not found Sep 08 10:18:11.856822 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:11.856782 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:11.857289 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:11.856878 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:11.857289 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:11.856962 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 10:18:11.857289 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:11.857019 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle podName:7484ddef-5576-43fe-857a-000cb91c4460 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:15.856991222 +0000 UTC m=+72.226256514 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle") pod "router-default-64bb84f4d-q4pg8" (UID: "7484ddef-5576-43fe-857a-000cb91c4460") : configmap references non-existent config key: service-ca.crt Sep 08 10:18:11.857289 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:11.857040 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs podName:7484ddef-5576-43fe-857a-000cb91c4460 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:15.857030929 +0000 UTC m=+72.226296216 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs") pod "router-default-64bb84f4d-q4pg8" (UID: "7484ddef-5576-43fe-857a-000cb91c4460") : secret "router-metrics-certs-default" not found Sep 08 10:18:13.368916 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:13.368874 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:18:13.369351 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:13.369056 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 10:18:13.369351 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:13.369075 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59557ddf67-5k75j: secret "image-registry-tls" not found Sep 08 10:18:13.369351 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:13.369138 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls podName:2dd92822-b06b-4eaa-8cf7-cc210c5de102 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:45.369117987 +0000 UTC m=+101.738383279 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls") pod "image-registry-59557ddf67-5k75j" (UID: "2dd92822-b06b-4eaa-8cf7-cc210c5de102") : secret "image-registry-tls" not found Sep 08 10:18:13.470281 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:13.470232 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:18:13.470474 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:13.470397 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 10:18:13.470565 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:13.470479 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls podName:d3689cda-0bc7-4455-8530-c685d3a0fe94 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:45.470458388 +0000 UTC m=+101.839723683 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls") pod "dns-default-8nhg6" (UID: "d3689cda-0bc7-4455-8530-c685d3a0fe94") : secret "dns-default-metrics-tls" not found Sep 08 10:18:14.403600 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.403490 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xlksv" event={"ID":"0d5943f3-ba9e-4dc9-91c1-9787f83d5305","Type":"ContainerStarted","Data":"00dea03154fa072288b5e212579bf916effa30d9e06dab84f89e7cae70132bb8"} Sep 08 10:18:14.404016 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.403608 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:18:14.404923 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.404900 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-56snn" event={"ID":"e0ef2303-d3cc-4560-b248-8d0c1b6e94e2","Type":"ContainerStarted","Data":"2f878a9f0dd1ca1ee8a84489a2cde495df85719e4ffba1c8fd6f0cdf557ff1fc"} Sep 08 10:18:14.437788 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.437738 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-56snn" podStartSLOduration=64.564680443 podStartE2EDuration="1m9.437722624s" totalImagesPullingTime="4.873042181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:18:14.437226375 +0000 UTC m=+70.806491673" watchObservedRunningTime="2026-09-08 10:18:14.437722624 +0000 UTC m=+70.806987921" Sep 08 10:18:14.437939 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.437921 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xlksv" podStartSLOduration=64.593574516 podStartE2EDuration="1m9.437915546s" totalImagesPullingTime="4.84434103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:18:14.420998516 +0000 UTC m=+70.790263814" watchObservedRunningTime="2026-09-08 10:18:14.437915546 +0000 UTC m=+70.807180843" Sep 08 10:18:14.821021 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.820986 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jxz2z"] Sep 08 10:18:14.823018 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.823003 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:14.825171 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.825148 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 08 10:18:14.825171 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.825163 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 08 10:18:14.825350 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.825275 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2wq7c\"" Sep 08 10:18:14.825395 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.825377 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 08 10:18:14.825480 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.825464 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 08 10:18:14.836211 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.836190 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jxz2z"] Sep 08 10:18:14.882017 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.881976 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-crio-socket\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:14.882017 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.882021 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:14.882220 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.882046 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkgd9\" (UniqueName: \"kubernetes.io/projected/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-api-access-tkgd9\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:14.882220 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.882065 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-data-volume\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:14.882220 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.882139 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:14.982773 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.982727 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:14.982942 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.982820 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-crio-socket\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:14.982942 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.982836 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:14.982942 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:14.982844 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 10:18:14.982942 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:14.982906 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm podName:4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:15.482889398 +0000 UTC m=+71.852154673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxz2z" (UID: "4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80") : configmap "kube-rbac-proxy" not found Sep 08 10:18:14.982942 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.982854 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkgd9\" (UniqueName: \"kubernetes.io/projected/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-api-access-tkgd9\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:14.982942 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.982938 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-crio-socket\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:14.983217 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:14.982969 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 10:18:14.983217 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.982986 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-data-volume\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:14.983217 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:14.983030 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls podName:4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:15.483014686 +0000 UTC m=+71.852279960 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxz2z" (UID: "4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80") : secret "insights-runtime-extractor-tls" not found Sep 08 10:18:14.983313 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:14.983234 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-data-volume\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:15.011107 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:15.011084 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkgd9\" (UniqueName: \"kubernetes.io/projected/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-api-access-tkgd9\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:15.487878 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:15.487840 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:15.488320 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:15.487903 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:15.488320 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:15.487980 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 10:18:15.488320 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:15.488019 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 10:18:15.488320 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:15.488054 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm podName:4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:16.488033997 +0000 UTC m=+72.857299295 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxz2z" (UID: "4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80") : configmap "kube-rbac-proxy" not found Sep 08 10:18:15.488320 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:15.488079 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls podName:4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:16.488062614 +0000 UTC m=+72.857327889 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxz2z" (UID: "4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80") : secret "insights-runtime-extractor-tls" not found Sep 08 10:18:15.891231 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:15.891150 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:15.891341 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:15.891240 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:15.891341 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:15.891305 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 10:18:15.891434 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:15.891345 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle podName:7484ddef-5576-43fe-857a-000cb91c4460 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:23.891331872 +0000 UTC m=+80.260597159 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle") pod "router-default-64bb84f4d-q4pg8" (UID: "7484ddef-5576-43fe-857a-000cb91c4460") : configmap references non-existent config key: service-ca.crt Sep 08 10:18:15.891434 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:15.891360 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs podName:7484ddef-5576-43fe-857a-000cb91c4460 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:23.891353832 +0000 UTC m=+80.260619107 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs") pod "router-default-64bb84f4d-q4pg8" (UID: "7484ddef-5576-43fe-857a-000cb91c4460") : secret "router-metrics-certs-default" not found Sep 08 10:18:16.497233 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:16.497188 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:16.497743 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:16.497282 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:16.497743 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:16.497317 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 10:18:16.497743 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:16.497385 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 10:18:16.497743 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:16.497400 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm podName:4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:18.497378352 +0000 UTC m=+74.866643641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxz2z" (UID: "4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80") : configmap "kube-rbac-proxy" not found Sep 08 10:18:16.497743 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:16.497426 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls podName:4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:18.497413719 +0000 UTC m=+74.866678994 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxz2z" (UID: "4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80") : secret "insights-runtime-extractor-tls" not found Sep 08 10:18:17.885674 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:17.885644 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lw9vz_e9f96208-ea87-4822-ae72-9c07e5d5fa8f/dns-node-resolver/0.log" Sep 08 10:18:18.482875 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:18.482846 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pv25g_919f3306-bc79-43be-92f0-2b46f301d6f7/node-ca/0.log" Sep 08 10:18:18.513269 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:18.513235 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:18.513369 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:18.513333 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:18.513369 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:18.513349 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 10:18:18.513437 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:18.513406 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm podName:4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:22.513391367 +0000 UTC m=+78.882656661 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxz2z" (UID: "4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80") : configmap "kube-rbac-proxy" not found Sep 08 10:18:18.513478 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:18.513450 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 10:18:18.513516 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:18.513505 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls podName:4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:22.513490328 +0000 UTC m=+78.882755620 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxz2z" (UID: "4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80") : secret "insights-runtime-extractor-tls" not found Sep 08 10:18:19.085259 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:19.085231 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8dfxz_3c246116-fdc5-4fde-8f72-8835d9718f7d/serve-healthcheck-canary/0.log" Sep 08 10:18:22.542845 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:22.542811 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:22.543313 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:22.542873 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:22.543313 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:22.542956 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 10:18:22.543313 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:22.542968 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 10:18:22.543313 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:22.543008 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm podName:4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:30.542994947 +0000 UTC m=+86.912260221 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxz2z" (UID: "4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80") : configmap "kube-rbac-proxy" not found Sep 08 10:18:22.543313 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:18:22.543020 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls podName:4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80 nodeName:}" failed. No retries permitted until 2026-09-08 10:18:30.543014759 +0000 UTC m=+86.912280033 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxz2z" (UID: "4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80") : secret "insights-runtime-extractor-tls" not found Sep 08 10:18:23.953636 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:23.953592 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:23.954039 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:23.953654 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:23.954216 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:23.954193 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7484ddef-5576-43fe-857a-000cb91c4460-service-ca-bundle\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:23.955911 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:23.955890 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7484ddef-5576-43fe-857a-000cb91c4460-metrics-certs\") pod \"router-default-64bb84f4d-q4pg8\" (UID: \"7484ddef-5576-43fe-857a-000cb91c4460\") " pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:24.018138 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:24.018102 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:24.291216 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:24.291181 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64bb84f4d-q4pg8"] Sep 08 10:18:24.296128 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:18:24.296100 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7484ddef_5576_43fe_857a_000cb91c4460.slice/crio-7c7a869be1a14485be42858017ad84c6653cb383805bdac3bf98c7d7655d359b WatchSource:0}: Error finding container 7c7a869be1a14485be42858017ad84c6653cb383805bdac3bf98c7d7655d359b: Status 404 returned error can't find the container with id 7c7a869be1a14485be42858017ad84c6653cb383805bdac3bf98c7d7655d359b Sep 08 10:18:24.431199 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:24.431159 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64bb84f4d-q4pg8" event={"ID":"7484ddef-5576-43fe-857a-000cb91c4460","Type":"ContainerStarted","Data":"243751464a959743a5268ff919013aa3301f1888df19cba73828856585a860a0"} Sep 08 10:18:24.431386 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:24.431214 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64bb84f4d-q4pg8" event={"ID":"7484ddef-5576-43fe-857a-000cb91c4460","Type":"ContainerStarted","Data":"7c7a869be1a14485be42858017ad84c6653cb383805bdac3bf98c7d7655d359b"} Sep 08 10:18:24.462437 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:24.462387 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-64bb84f4d-q4pg8" podStartSLOduration=16.462372173 podStartE2EDuration="16.462372173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:18:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:18:24.462266054 +0000 UTC m=+80.831531351" watchObservedRunningTime="2026-09-08 10:18:24.462372173 +0000 UTC m=+80.831637460" Sep 08 10:18:25.018615 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:25.018576 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:25.021164 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:25.021142 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:25.433673 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:25.433592 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:25.434861 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:25.434837 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-64bb84f4d-q4pg8" Sep 08 10:18:30.603261 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:30.603110 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:30.603261 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:30.603200 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:30.603690 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:30.603668 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:30.605409 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:30.605385 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxz2z\" (UID: \"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80\") " pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:30.732473 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:30.732430 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jxz2z" Sep 08 10:18:31.001392 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:31.001362 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jxz2z"] Sep 08 10:18:31.004150 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:18:31.004124 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e9e96ff_b26f_40f3_ab4c_5a1bd4115a80.slice/crio-bba3286846bbfc9e70097e9f0c3cffc279ac99469fd59d6f26954f1b8e4ed0a0 WatchSource:0}: Error finding container bba3286846bbfc9e70097e9f0c3cffc279ac99469fd59d6f26954f1b8e4ed0a0: Status 404 returned error can't find the container with id bba3286846bbfc9e70097e9f0c3cffc279ac99469fd59d6f26954f1b8e4ed0a0 Sep 08 10:18:31.449206 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:31.449126 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jxz2z" event={"ID":"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80","Type":"ContainerStarted","Data":"e290a88d840f0473c0426e5cbe55ab6c08a7127a3380e2cc531ea1ec2ac397a4"} Sep 08 10:18:31.449206 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:31.449183 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jxz2z" event={"ID":"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80","Type":"ContainerStarted","Data":"bba3286846bbfc9e70097e9f0c3cffc279ac99469fd59d6f26954f1b8e4ed0a0"} Sep 08 10:18:32.457249 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:32.457214 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jxz2z" event={"ID":"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80","Type":"ContainerStarted","Data":"a330d6298e6b7a344e31477deb394562013aa8f9302d8d4c2ca4bf5094ba21cd"} Sep 08 10:18:34.467281 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:34.467240 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jxz2z" event={"ID":"4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80","Type":"ContainerStarted","Data":"578e9b78870a0f2585409d043eb18013b282e884c118bd8d4544f3ebd4eb104b"} Sep 08 10:18:34.496764 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:34.496710 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jxz2z" podStartSLOduration=18.186838113 podStartE2EDuration="20.49669529s" totalImagesPullingTime="2.309857177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:18:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:18:34.495678618 +0000 UTC m=+90.864943916" watchObservedRunningTime="2026-09-08 10:18:34.49669529 +0000 UTC m=+90.865960586" Sep 08 10:18:35.157197 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:35.157162 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-7gv99"] Sep 08 10:18:35.160119 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:35.160104 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-7gv99" Sep 08 10:18:35.162213 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:35.162189 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 08 10:18:35.162342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:35.162289 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-mvv5w\"" Sep 08 10:18:35.162342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:35.162294 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 08 10:18:35.176462 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:35.176433 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-7gv99"] Sep 08 10:18:35.240203 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:35.240162 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-456vx\" (UniqueName: \"kubernetes.io/projected/d605b050-e240-493c-8692-ca923917cdd6-kube-api-access-456vx\") pod \"downloads-85d98c998f-7gv99\" (UID: \"d605b050-e240-493c-8692-ca923917cdd6\") " pod="openshift-console/downloads-85d98c998f-7gv99" Sep 08 10:18:35.340702 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:35.340616 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-456vx\" (UniqueName: \"kubernetes.io/projected/d605b050-e240-493c-8692-ca923917cdd6-kube-api-access-456vx\") pod \"downloads-85d98c998f-7gv99\" (UID: \"d605b050-e240-493c-8692-ca923917cdd6\") " pod="openshift-console/downloads-85d98c998f-7gv99" Sep 08 10:18:35.351571 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:35.351516 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-456vx\" (UniqueName: \"kubernetes.io/projected/d605b050-e240-493c-8692-ca923917cdd6-kube-api-access-456vx\") pod \"downloads-85d98c998f-7gv99\" (UID: \"d605b050-e240-493c-8692-ca923917cdd6\") " pod="openshift-console/downloads-85d98c998f-7gv99" Sep 08 10:18:35.468591 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:35.468562 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-7gv99" Sep 08 10:18:35.739178 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:35.739148 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-7gv99"] Sep 08 10:18:35.741988 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:18:35.741956 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd605b050_e240_493c_8692_ca923917cdd6.slice/crio-fe4f8764b6f00d652f2d6f2f0222070b1625a900c55fbf72ccff6bf4490aa7c5 WatchSource:0}: Error finding container fe4f8764b6f00d652f2d6f2f0222070b1625a900c55fbf72ccff6bf4490aa7c5: Status 404 returned error can't find the container with id fe4f8764b6f00d652f2d6f2f0222070b1625a900c55fbf72ccff6bf4490aa7c5 Sep 08 10:18:36.475587 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:36.475551 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-7gv99" event={"ID":"d605b050-e240-493c-8692-ca923917cdd6","Type":"ContainerStarted","Data":"fe4f8764b6f00d652f2d6f2f0222070b1625a900c55fbf72ccff6bf4490aa7c5"} Sep 08 10:18:41.685629 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.685592 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c6cc7594-sqsnv"] Sep 08 10:18:41.688743 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.688720 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.691986 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.691781 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 08 10:18:41.691986 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.691798 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 08 10:18:41.691986 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.691806 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 08 10:18:41.691986 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.691805 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gvqg2\"" Sep 08 10:18:41.691986 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.691899 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 08 10:18:41.691986 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.691959 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 08 10:18:41.700152 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.700128 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c6cc7594-sqsnv"] Sep 08 10:18:41.796145 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.796105 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-service-ca\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.796332 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.796191 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7156d04a-f799-4bea-a216-6a2a846cef60-console-oauth-config\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.796332 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.796225 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx9nw\" (UniqueName: \"kubernetes.io/projected/7156d04a-f799-4bea-a216-6a2a846cef60-kube-api-access-rx9nw\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.796332 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.796262 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-oauth-serving-cert\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.796332 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.796291 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7156d04a-f799-4bea-a216-6a2a846cef60-console-serving-cert\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.796548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.796466 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-console-config\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.897345 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.897304 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7156d04a-f799-4bea-a216-6a2a846cef60-console-oauth-config\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.897559 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.897352 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx9nw\" (UniqueName: \"kubernetes.io/projected/7156d04a-f799-4bea-a216-6a2a846cef60-kube-api-access-rx9nw\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.897559 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.897388 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-oauth-serving-cert\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.897559 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.897409 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7156d04a-f799-4bea-a216-6a2a846cef60-console-serving-cert\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.897559 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.897485 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-console-config\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.897559 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.897517 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-service-ca\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.898262 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.898233 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-oauth-serving-cert\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.898362 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.898309 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-service-ca\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.898414 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.898396 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-console-config\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.900065 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.900039 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7156d04a-f799-4bea-a216-6a2a846cef60-console-oauth-config\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.900215 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.900193 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7156d04a-f799-4bea-a216-6a2a846cef60-console-serving-cert\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:41.906684 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:41.906657 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx9nw\" (UniqueName: \"kubernetes.io/projected/7156d04a-f799-4bea-a216-6a2a846cef60-kube-api-access-rx9nw\") pod \"console-6c6cc7594-sqsnv\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:42.000049 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:42.000011 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:18:42.285542 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:42.285444 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c6cc7594-sqsnv"] Sep 08 10:18:42.288958 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:18:42.288915 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7156d04a_f799_4bea_a216_6a2a846cef60.slice/crio-53f32f018c4715eafb1e55b137564b8433fe90f3d7a6f2510b650c0420fced65 WatchSource:0}: Error finding container 53f32f018c4715eafb1e55b137564b8433fe90f3d7a6f2510b650c0420fced65: Status 404 returned error can't find the container with id 53f32f018c4715eafb1e55b137564b8433fe90f3d7a6f2510b650c0420fced65 Sep 08 10:18:42.497332 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:42.497288 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6cc7594-sqsnv" event={"ID":"7156d04a-f799-4bea-a216-6a2a846cef60","Type":"ContainerStarted","Data":"53f32f018c4715eafb1e55b137564b8433fe90f3d7a6f2510b650c0420fced65"} Sep 08 10:18:45.280619 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.280581 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kzzx6"] Sep 08 10:18:45.287000 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.286981 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kzzx6" Sep 08 10:18:45.289111 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.289089 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 08 10:18:45.289206 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.289188 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-hgx7v\"" Sep 08 10:18:45.295759 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.295738 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kzzx6"] Sep 08 10:18:45.410981 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.410943 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xlksv" Sep 08 10:18:45.431427 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.431391 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c9ce2aaf-a9c4-433c-9747-40228ed457ab-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-kzzx6\" (UID: \"c9ce2aaf-a9c4-433c-9747-40228ed457ab\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kzzx6" Sep 08 10:18:45.431626 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.431491 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:18:45.434146 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.434122 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls\") pod \"image-registry-59557ddf67-5k75j\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:18:45.458673 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.458644 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-svksl\"" Sep 08 10:18:45.466955 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.466916 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:18:45.532077 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.531975 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c9ce2aaf-a9c4-433c-9747-40228ed457ab-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-kzzx6\" (UID: \"c9ce2aaf-a9c4-433c-9747-40228ed457ab\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kzzx6" Sep 08 10:18:45.532077 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.532030 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:18:45.534604 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.534577 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3689cda-0bc7-4455-8530-c685d3a0fe94-metrics-tls\") pod \"dns-default-8nhg6\" (UID: \"d3689cda-0bc7-4455-8530-c685d3a0fe94\") " pod="openshift-dns/dns-default-8nhg6" Sep 08 10:18:45.534604 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.534594 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c9ce2aaf-a9c4-433c-9747-40228ed457ab-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-kzzx6\" (UID: \"c9ce2aaf-a9c4-433c-9747-40228ed457ab\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kzzx6" Sep 08 10:18:45.596381 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.596341 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kzzx6" Sep 08 10:18:45.785299 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.785207 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f7bmn\"" Sep 08 10:18:45.793142 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:45.793108 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8nhg6" Sep 08 10:18:51.294259 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.294226 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-547759fdf4-7blpv"] Sep 08 10:18:51.296854 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.296835 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.305239 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.305218 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 08 10:18:51.308809 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.308784 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-547759fdf4-7blpv"] Sep 08 10:18:51.382974 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.382938 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-oauth-serving-cert\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.383145 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.382977 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-config\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.383145 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.383021 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-serving-cert\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.383145 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.383046 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72mp6\" (UniqueName: \"kubernetes.io/projected/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-kube-api-access-72mp6\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.383145 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.383076 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-service-ca\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.383345 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.383190 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-trusted-ca-bundle\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.383345 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.383233 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-oauth-config\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.484466 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.484422 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-oauth-config\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.484659 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.484485 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-oauth-serving-cert\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.484659 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.484509 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-config\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.484659 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.484551 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-serving-cert\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.484659 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.484576 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72mp6\" (UniqueName: \"kubernetes.io/projected/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-kube-api-access-72mp6\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.484659 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.484615 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-service-ca\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.484912 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.484666 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-trusted-ca-bundle\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.485389 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.485353 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-oauth-serving-cert\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.485521 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.485472 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-service-ca\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.485521 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.485499 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-config\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.485662 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.485609 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-trusted-ca-bundle\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.487136 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.487113 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-oauth-config\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.487418 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.487398 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-serving-cert\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.496108 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.496081 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72mp6\" (UniqueName: \"kubernetes.io/projected/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-kube-api-access-72mp6\") pod \"console-547759fdf4-7blpv\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:51.607353 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:51.607271 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:18:52.468101 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:52.468015 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kzzx6"] Sep 08 10:18:52.471667 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:52.471303 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8nhg6"] Sep 08 10:18:52.473616 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:52.473416 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-547759fdf4-7blpv"] Sep 08 10:18:52.474351 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:18:52.474321 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9ce2aaf_a9c4_433c_9747_40228ed457ab.slice/crio-4d2c2c79ef8ee5de319d76a63503bf88b6ca3f4ae5ed39b3e29f99f7dca605dc WatchSource:0}: Error finding container 4d2c2c79ef8ee5de319d76a63503bf88b6ca3f4ae5ed39b3e29f99f7dca605dc: Status 404 returned error can't find the container with id 4d2c2c79ef8ee5de319d76a63503bf88b6ca3f4ae5ed39b3e29f99f7dca605dc Sep 08 10:18:52.475629 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:52.475518 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59557ddf67-5k75j"] Sep 08 10:18:52.476843 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:18:52.476751 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3689cda_0bc7_4455_8530_c685d3a0fe94.slice/crio-a0169d5a3c896357833b2aa223b41c3ed3a4ea266890b6d370d1ebb37f970257 WatchSource:0}: Error finding container a0169d5a3c896357833b2aa223b41c3ed3a4ea266890b6d370d1ebb37f970257: Status 404 returned error can't find the container with id a0169d5a3c896357833b2aa223b41c3ed3a4ea266890b6d370d1ebb37f970257 Sep 08 10:18:52.477283 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:18:52.477258 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc1f1e0d_5b89_4e05_aaf0_549dd3b604fa.slice/crio-3482d7be8c2138ac9fcdfc401cba93be2dabba84242983768da2f694b8824743 WatchSource:0}: Error finding container 3482d7be8c2138ac9fcdfc401cba93be2dabba84242983768da2f694b8824743: Status 404 returned error can't find the container with id 3482d7be8c2138ac9fcdfc401cba93be2dabba84242983768da2f694b8824743 Sep 08 10:18:52.478306 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:18:52.478271 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dd92822_b06b_4eaa_8cf7_cc210c5de102.slice/crio-519bda657f76dda313289cadacbcd0f88ce70821d673eea8d49e8e329d8792d6 WatchSource:0}: Error finding container 519bda657f76dda313289cadacbcd0f88ce70821d673eea8d49e8e329d8792d6: Status 404 returned error can't find the container with id 519bda657f76dda313289cadacbcd0f88ce70821d673eea8d49e8e329d8792d6 Sep 08 10:18:52.537665 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:52.537638 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-7gv99" event={"ID":"d605b050-e240-493c-8692-ca923917cdd6","Type":"ContainerStarted","Data":"4bf0846fcbb0c3e8338978070480c2c9a0dc8dea0477ae731089b2fc6c811781"} Sep 08 10:18:52.537823 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:52.537811 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-7gv99" Sep 08 10:18:52.538973 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:52.538949 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kzzx6" event={"ID":"c9ce2aaf-a9c4-433c-9747-40228ed457ab","Type":"ContainerStarted","Data":"4d2c2c79ef8ee5de319d76a63503bf88b6ca3f4ae5ed39b3e29f99f7dca605dc"} Sep 08 10:18:52.543695 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:52.543261 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-547759fdf4-7blpv" event={"ID":"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa","Type":"ContainerStarted","Data":"3482d7be8c2138ac9fcdfc401cba93be2dabba84242983768da2f694b8824743"} Sep 08 10:18:52.545733 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:52.544080 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-7gv99" Sep 08 10:18:52.548070 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:52.548041 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6cc7594-sqsnv" event={"ID":"7156d04a-f799-4bea-a216-6a2a846cef60","Type":"ContainerStarted","Data":"75a6cb5ee591ec57a632194d9c7dbd09489ae86c79ecf57eee7d59ab93c9ce23"} Sep 08 10:18:52.549709 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:52.549680 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8nhg6" event={"ID":"d3689cda-0bc7-4455-8530-c685d3a0fe94","Type":"ContainerStarted","Data":"a0169d5a3c896357833b2aa223b41c3ed3a4ea266890b6d370d1ebb37f970257"} Sep 08 10:18:52.551296 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:52.551240 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59557ddf67-5k75j" event={"ID":"2dd92822-b06b-4eaa-8cf7-cc210c5de102","Type":"ContainerStarted","Data":"519bda657f76dda313289cadacbcd0f88ce70821d673eea8d49e8e329d8792d6"} Sep 08 10:18:52.588911 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:52.588390 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-7gv99" podStartSLOduration=1.079686322 podStartE2EDuration="17.588366882s" totalImagesPullingTime="16.50868056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:18:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:18:52.562832926 +0000 UTC m=+108.932098226" watchObservedRunningTime="2026-09-08 10:18:52.588366882 +0000 UTC m=+108.957632181" Sep 08 10:18:53.559633 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:53.559592 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59557ddf67-5k75j" event={"ID":"2dd92822-b06b-4eaa-8cf7-cc210c5de102","Type":"ContainerStarted","Data":"6e4edac8971bc3d8c8b9b5aec7b8a9890b1b3b0be7683d834e95666894458dfa"} Sep 08 10:18:53.560101 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:53.559706 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:18:53.562397 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:53.562366 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-547759fdf4-7blpv" event={"ID":"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa","Type":"ContainerStarted","Data":"d8d05118de3bf4adde8957a8d2d0fe1fa577b9d796baa1d3592f59b5d358ea59"} Sep 08 10:18:53.587126 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:53.587062 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c6cc7594-sqsnv" podStartSLOduration=2.692475958 podStartE2EDuration="12.587041872s" totalImagesPullingTime="9.894565914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:18:52.608939563 +0000 UTC m=+108.978204862" watchObservedRunningTime="2026-09-08 10:18:53.587041872 +0000 UTC m=+109.956307174" Sep 08 10:18:53.587962 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:53.587385 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59557ddf67-5k75j" podStartSLOduration=108.587373083 podStartE2EDuration="1m48.587373083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:17:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:18:53.586599013 +0000 UTC m=+109.955864311" watchObservedRunningTime="2026-09-08 10:18:53.587373083 +0000 UTC m=+109.956638383" Sep 08 10:18:53.610248 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:53.609967 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-547759fdf4-7blpv" podStartSLOduration=2.60994517 podStartE2EDuration="2.60994517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:18:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:18:53.60847433 +0000 UTC m=+109.977739631" watchObservedRunningTime="2026-09-08 10:18:53.60994517 +0000 UTC m=+109.979210468" Sep 08 10:18:55.573589 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:55.573481 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8nhg6" event={"ID":"d3689cda-0bc7-4455-8530-c685d3a0fe94","Type":"ContainerStarted","Data":"54d3ecfcc314d6b2abdeee1e1b9f08b09afeea1dd2bd962935803cc80b566690"} Sep 08 10:18:55.574038 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:55.573609 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8nhg6" Sep 08 10:18:55.574038 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:55.573636 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8nhg6" event={"ID":"d3689cda-0bc7-4455-8530-c685d3a0fe94","Type":"ContainerStarted","Data":"4d31dbf864fa43f06d0f8cd75d7a687d5806b7c47f94901b72fd4510bcd8513d"} Sep 08 10:18:55.575749 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:55.575715 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kzzx6" event={"ID":"c9ce2aaf-a9c4-433c-9747-40228ed457ab","Type":"ContainerStarted","Data":"01200d1f09a46de81d83e69ed27ef65a51138d24f506a409b28e05906c25eee8"} Sep 08 10:18:55.575937 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:55.575914 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kzzx6" Sep 08 10:18:55.581421 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:55.581398 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kzzx6" Sep 08 10:18:55.596700 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:55.596650 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8nhg6" podStartSLOduration=72.066946642 podStartE2EDuration="1m14.59663772s" totalImagesPullingTime="2.529691078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:18:55.595406868 +0000 UTC m=+111.964672166" watchObservedRunningTime="2026-09-08 10:18:55.59663772 +0000 UTC m=+111.965903017" Sep 08 10:18:55.615268 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:55.615214 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kzzx6" podStartSLOduration=8.089260907 podStartE2EDuration="10.615196287s" totalImagesPullingTime="2.52593538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:18:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:18:55.614139176 +0000 UTC m=+111.983404479" watchObservedRunningTime="2026-09-08 10:18:55.615196287 +0000 UTC m=+111.984461583" Sep 08 10:18:56.372795 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.372764 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-c9cp8"] Sep 08 10:18:56.411803 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.411765 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-c9cp8"] Sep 08 10:18:56.411960 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.411908 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" Sep 08 10:18:56.414569 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.414517 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 08 10:18:56.414716 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.414605 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 08 10:18:56.415298 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.415270 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 08 10:18:56.415432 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.415314 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-wdg6t\"" Sep 08 10:18:56.415432 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.415355 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 08 10:18:56.415654 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.415635 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 08 10:18:56.532149 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.532098 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fd9e1e2-cfae-4be8-a82c-bea62267a9ad-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-c9cp8\" (UID: \"6fd9e1e2-cfae-4be8-a82c-bea62267a9ad\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" Sep 08 10:18:56.532328 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.532218 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6fd9e1e2-cfae-4be8-a82c-bea62267a9ad-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-c9cp8\" (UID: \"6fd9e1e2-cfae-4be8-a82c-bea62267a9ad\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" Sep 08 10:18:56.532328 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.532256 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6fd9e1e2-cfae-4be8-a82c-bea62267a9ad-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-c9cp8\" (UID: \"6fd9e1e2-cfae-4be8-a82c-bea62267a9ad\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" Sep 08 10:18:56.532425 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.532321 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qh7k\" (UniqueName: \"kubernetes.io/projected/6fd9e1e2-cfae-4be8-a82c-bea62267a9ad-kube-api-access-9qh7k\") pod \"prometheus-operator-6f4946db56-c9cp8\" (UID: \"6fd9e1e2-cfae-4be8-a82c-bea62267a9ad\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" Sep 08 10:18:56.633032 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.632947 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6fd9e1e2-cfae-4be8-a82c-bea62267a9ad-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-c9cp8\" (UID: \"6fd9e1e2-cfae-4be8-a82c-bea62267a9ad\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" Sep 08 10:18:56.633032 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.633024 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qh7k\" (UniqueName: \"kubernetes.io/projected/6fd9e1e2-cfae-4be8-a82c-bea62267a9ad-kube-api-access-9qh7k\") pod \"prometheus-operator-6f4946db56-c9cp8\" (UID: \"6fd9e1e2-cfae-4be8-a82c-bea62267a9ad\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" Sep 08 10:18:56.633578 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.633067 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fd9e1e2-cfae-4be8-a82c-bea62267a9ad-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-c9cp8\" (UID: \"6fd9e1e2-cfae-4be8-a82c-bea62267a9ad\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" Sep 08 10:18:56.633578 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.633142 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6fd9e1e2-cfae-4be8-a82c-bea62267a9ad-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-c9cp8\" (UID: \"6fd9e1e2-cfae-4be8-a82c-bea62267a9ad\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" Sep 08 10:18:56.634172 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.634139 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6fd9e1e2-cfae-4be8-a82c-bea62267a9ad-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-c9cp8\" (UID: \"6fd9e1e2-cfae-4be8-a82c-bea62267a9ad\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" Sep 08 10:18:56.636118 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.635906 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6fd9e1e2-cfae-4be8-a82c-bea62267a9ad-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-c9cp8\" (UID: \"6fd9e1e2-cfae-4be8-a82c-bea62267a9ad\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" Sep 08 10:18:56.636118 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.635929 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6fd9e1e2-cfae-4be8-a82c-bea62267a9ad-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-c9cp8\" (UID: \"6fd9e1e2-cfae-4be8-a82c-bea62267a9ad\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" Sep 08 10:18:56.642046 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.642023 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qh7k\" (UniqueName: \"kubernetes.io/projected/6fd9e1e2-cfae-4be8-a82c-bea62267a9ad-kube-api-access-9qh7k\") pod \"prometheus-operator-6f4946db56-c9cp8\" (UID: \"6fd9e1e2-cfae-4be8-a82c-bea62267a9ad\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" Sep 08 10:18:56.722859 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:56.722824 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" Sep 08 10:18:57.031864 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:57.031830 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-c9cp8"] Sep 08 10:18:57.035233 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:18:57.035193 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fd9e1e2_cfae_4be8_a82c_bea62267a9ad.slice/crio-b5aa6d0b7631f5a9215ae227c525567bd83c30ba24d8079fb7a3314931c37865 WatchSource:0}: Error finding container b5aa6d0b7631f5a9215ae227c525567bd83c30ba24d8079fb7a3314931c37865: Status 404 returned error can't find the container with id b5aa6d0b7631f5a9215ae227c525567bd83c30ba24d8079fb7a3314931c37865 Sep 08 10:18:57.586188 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:57.586117 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" event={"ID":"6fd9e1e2-cfae-4be8-a82c-bea62267a9ad","Type":"ContainerStarted","Data":"b5aa6d0b7631f5a9215ae227c525567bd83c30ba24d8079fb7a3314931c37865"} Sep 08 10:18:57.721916 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:57.721506 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59557ddf67-5k75j"] Sep 08 10:18:59.595456 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:59.595408 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" event={"ID":"6fd9e1e2-cfae-4be8-a82c-bea62267a9ad","Type":"ContainerStarted","Data":"a53a6afc00d5efe5527fdfce7bc5efbbebd2589fd83a67576383764b7fbd08ff"} Sep 08 10:18:59.596007 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:59.595506 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" event={"ID":"6fd9e1e2-cfae-4be8-a82c-bea62267a9ad","Type":"ContainerStarted","Data":"f422c309d0f89cc8971e682e96047d58e4f80efefed07c19c48494131f886948"} Sep 08 10:18:59.614381 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:18:59.614329 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-c9cp8" podStartSLOduration=1.763020179 podStartE2EDuration="3.614312784s" totalImagesPullingTime="1.851292605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:18:59.61315711 +0000 UTC m=+115.982422408" watchObservedRunningTime="2026-09-08 10:18:59.614312784 +0000 UTC m=+115.983578080" Sep 08 10:19:01.607748 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.607717 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:19:01.608189 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.607772 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:19:01.612294 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.612274 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:19:01.739261 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.739236 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc"] Sep 08 10:19:01.771913 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.771889 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2"] Sep 08 10:19:01.772012 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.771998 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" Sep 08 10:19:01.773864 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.773847 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-c4p55\"" Sep 08 10:19:01.774008 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.773909 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 08 10:19:01.774104 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.774088 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 08 10:19:01.778673 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.778268 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llxkm\" (UniqueName: \"kubernetes.io/projected/97778083-73ca-43eb-9454-1d3a28e8528a-kube-api-access-llxkm\") pod \"openshift-state-metrics-657fd6c54-t7bpc\" (UID: \"97778083-73ca-43eb-9454-1d3a28e8528a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" Sep 08 10:19:01.778673 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.778353 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97778083-73ca-43eb-9454-1d3a28e8528a-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-t7bpc\" (UID: \"97778083-73ca-43eb-9454-1d3a28e8528a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" Sep 08 10:19:01.778673 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.778397 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97778083-73ca-43eb-9454-1d3a28e8528a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-t7bpc\" (UID: \"97778083-73ca-43eb-9454-1d3a28e8528a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" Sep 08 10:19:01.778673 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.778429 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97778083-73ca-43eb-9454-1d3a28e8528a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-t7bpc\" (UID: \"97778083-73ca-43eb-9454-1d3a28e8528a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" Sep 08 10:19:01.787916 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.787896 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc"] Sep 08 10:19:01.788016 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.787962 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2"] Sep 08 10:19:01.788016 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.787905 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.788016 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.787999 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8299s"] Sep 08 10:19:01.790316 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.790296 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 08 10:19:01.790476 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.790457 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-j2nbc\"" Sep 08 10:19:01.791016 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.791000 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 08 10:19:01.791119 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.791099 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 08 10:19:01.806032 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.806014 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.808158 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.808129 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 08 10:19:01.808239 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.808175 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 08 10:19:01.808239 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.808212 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 08 10:19:01.808379 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.808363 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9s5qq\"" Sep 08 10:19:01.879432 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879382 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llxkm\" (UniqueName: \"kubernetes.io/projected/97778083-73ca-43eb-9454-1d3a28e8528a-kube-api-access-llxkm\") pod \"openshift-state-metrics-657fd6c54-t7bpc\" (UID: \"97778083-73ca-43eb-9454-1d3a28e8528a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" Sep 08 10:19:01.879432 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879411 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3a437240-6ceb-4a5f-b994-4119492c928f-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.879589 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879445 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3a437240-6ceb-4a5f-b994-4119492c928f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.879589 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879463 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97778083-73ca-43eb-9454-1d3a28e8528a-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-t7bpc\" (UID: \"97778083-73ca-43eb-9454-1d3a28e8528a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" Sep 08 10:19:01.879589 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879485 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cf550904-a7ea-4cd1-8843-766b17279143-node-exporter-wtmp\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.879589 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879506 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf550904-a7ea-4cd1-8843-766b17279143-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.879589 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879545 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cf550904-a7ea-4cd1-8843-766b17279143-root\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.879589 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879570 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a437240-6ceb-4a5f-b994-4119492c928f-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.879861 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879644 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97778083-73ca-43eb-9454-1d3a28e8528a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-t7bpc\" (UID: \"97778083-73ca-43eb-9454-1d3a28e8528a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" Sep 08 10:19:01.879861 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879685 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lldw4\" (UniqueName: \"kubernetes.io/projected/cf550904-a7ea-4cd1-8843-766b17279143-kube-api-access-lldw4\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.879861 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879720 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97778083-73ca-43eb-9454-1d3a28e8528a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-t7bpc\" (UID: \"97778083-73ca-43eb-9454-1d3a28e8528a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" Sep 08 10:19:01.879861 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879753 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a437240-6ceb-4a5f-b994-4119492c928f-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.879861 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879787 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cf550904-a7ea-4cd1-8843-766b17279143-node-exporter-textfile\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.879861 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879851 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cf550904-a7ea-4cd1-8843-766b17279143-node-exporter-accelerators-collector-config\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.880122 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879872 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf550904-a7ea-4cd1-8843-766b17279143-metrics-client-ca\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.880122 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879890 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cf550904-a7ea-4cd1-8843-766b17279143-node-exporter-tls\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.880122 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879905 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf550904-a7ea-4cd1-8843-766b17279143-sys\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.880122 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879921 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78rpr\" (UniqueName: \"kubernetes.io/projected/3a437240-6ceb-4a5f-b994-4119492c928f-kube-api-access-78rpr\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.880122 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.879942 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a437240-6ceb-4a5f-b994-4119492c928f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.880122 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.880081 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97778083-73ca-43eb-9454-1d3a28e8528a-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-t7bpc\" (UID: \"97778083-73ca-43eb-9454-1d3a28e8528a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" Sep 08 10:19:01.881929 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.881907 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97778083-73ca-43eb-9454-1d3a28e8528a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-t7bpc\" (UID: \"97778083-73ca-43eb-9454-1d3a28e8528a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" Sep 08 10:19:01.882056 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.882035 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97778083-73ca-43eb-9454-1d3a28e8528a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-t7bpc\" (UID: \"97778083-73ca-43eb-9454-1d3a28e8528a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" Sep 08 10:19:01.888602 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.888577 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llxkm\" (UniqueName: \"kubernetes.io/projected/97778083-73ca-43eb-9454-1d3a28e8528a-kube-api-access-llxkm\") pod \"openshift-state-metrics-657fd6c54-t7bpc\" (UID: \"97778083-73ca-43eb-9454-1d3a28e8528a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" Sep 08 10:19:01.980834 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.980808 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a437240-6ceb-4a5f-b994-4119492c928f-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.980962 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.980843 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cf550904-a7ea-4cd1-8843-766b17279143-node-exporter-textfile\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.980962 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.980860 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cf550904-a7ea-4cd1-8843-766b17279143-node-exporter-accelerators-collector-config\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.981042 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.980968 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf550904-a7ea-4cd1-8843-766b17279143-metrics-client-ca\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.981042 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.981011 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cf550904-a7ea-4cd1-8843-766b17279143-node-exporter-tls\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.981144 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.981064 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf550904-a7ea-4cd1-8843-766b17279143-sys\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.981144 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.981103 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78rpr\" (UniqueName: \"kubernetes.io/projected/3a437240-6ceb-4a5f-b994-4119492c928f-kube-api-access-78rpr\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.981144 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.981104 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cf550904-a7ea-4cd1-8843-766b17279143-node-exporter-textfile\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.981284 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.981148 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf550904-a7ea-4cd1-8843-766b17279143-sys\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.981284 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.981140 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a437240-6ceb-4a5f-b994-4119492c928f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.981284 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.981227 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3a437240-6ceb-4a5f-b994-4119492c928f-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.981426 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.981291 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3a437240-6ceb-4a5f-b994-4119492c928f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.982002 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.981332 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cf550904-a7ea-4cd1-8843-766b17279143-node-exporter-wtmp\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.982002 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.981707 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf550904-a7ea-4cd1-8843-766b17279143-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.982002 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.981757 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cf550904-a7ea-4cd1-8843-766b17279143-root\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.982002 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.981794 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a437240-6ceb-4a5f-b994-4119492c928f-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.982002 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.981838 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lldw4\" (UniqueName: \"kubernetes.io/projected/cf550904-a7ea-4cd1-8843-766b17279143-kube-api-access-lldw4\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.982002 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.981890 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cf550904-a7ea-4cd1-8843-766b17279143-node-exporter-accelerators-collector-config\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.982002 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.981898 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf550904-a7ea-4cd1-8843-766b17279143-metrics-client-ca\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.982396 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.982211 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3a437240-6ceb-4a5f-b994-4119492c928f-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.982396 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.982266 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a437240-6ceb-4a5f-b994-4119492c928f-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.982520 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.982500 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cf550904-a7ea-4cd1-8843-766b17279143-node-exporter-wtmp\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.986322 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.982946 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cf550904-a7ea-4cd1-8843-766b17279143-root\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.986322 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.983176 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3a437240-6ceb-4a5f-b994-4119492c928f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.986322 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.984401 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a437240-6ceb-4a5f-b994-4119492c928f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.987177 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.986836 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a437240-6ceb-4a5f-b994-4119492c928f-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.987774 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.987752 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf550904-a7ea-4cd1-8843-766b17279143-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.987871 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.987813 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cf550904-a7ea-4cd1-8843-766b17279143-node-exporter-tls\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:01.995901 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.995881 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78rpr\" (UniqueName: \"kubernetes.io/projected/3a437240-6ceb-4a5f-b994-4119492c928f-kube-api-access-78rpr\") pod \"kube-state-metrics-74cb57df7-hjzj2\" (UID: \"3a437240-6ceb-4a5f-b994-4119492c928f\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:01.996206 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:01.996191 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lldw4\" (UniqueName: \"kubernetes.io/projected/cf550904-a7ea-4cd1-8843-766b17279143-kube-api-access-lldw4\") pod \"node-exporter-8299s\" (UID: \"cf550904-a7ea-4cd1-8843-766b17279143\") " pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:02.001190 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.001171 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:19:02.001261 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.001252 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:19:02.005749 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.005731 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:19:02.080829 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.080809 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" Sep 08 10:19:02.096086 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.096068 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" Sep 08 10:19:02.114513 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.114490 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8299s" Sep 08 10:19:02.167188 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:19:02.167144 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf550904_a7ea_4cd1_8843_766b17279143.slice/crio-e3fea0847121796a5ec677051a271d66c3bcc8240bf7f819ba3b6d1df8a08d11 WatchSource:0}: Error finding container e3fea0847121796a5ec677051a271d66c3bcc8240bf7f819ba3b6d1df8a08d11: Status 404 returned error can't find the container with id e3fea0847121796a5ec677051a271d66c3bcc8240bf7f819ba3b6d1df8a08d11 Sep 08 10:19:02.374043 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.374023 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc"] Sep 08 10:19:02.375997 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:19:02.375970 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97778083_73ca_43eb_9454_1d3a28e8528a.slice/crio-fdbaf904e01c99a57bffd7e90b0a9ecdff5221e83c9946787b8d152c6d83b3a5 WatchSource:0}: Error finding container fdbaf904e01c99a57bffd7e90b0a9ecdff5221e83c9946787b8d152c6d83b3a5: Status 404 returned error can't find the container with id fdbaf904e01c99a57bffd7e90b0a9ecdff5221e83c9946787b8d152c6d83b3a5 Sep 08 10:19:02.412439 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.412414 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2"] Sep 08 10:19:02.416182 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:19:02.416159 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a437240_6ceb_4a5f_b994_4119492c928f.slice/crio-d14b07073517ae00401420efafb146a33b0e62f39fab04bca8d2dc8930cfd0ef WatchSource:0}: Error finding container d14b07073517ae00401420efafb146a33b0e62f39fab04bca8d2dc8930cfd0ef: Status 404 returned error can't find the container with id d14b07073517ae00401420efafb146a33b0e62f39fab04bca8d2dc8930cfd0ef Sep 08 10:19:02.606592 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.606564 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" event={"ID":"97778083-73ca-43eb-9454-1d3a28e8528a","Type":"ContainerStarted","Data":"68c9334d3e21d06b670a642b1eb4eb71889df73141c5d253cfb79ceea0388b4e"} Sep 08 10:19:02.606689 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.606633 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" event={"ID":"97778083-73ca-43eb-9454-1d3a28e8528a","Type":"ContainerStarted","Data":"fdbaf904e01c99a57bffd7e90b0a9ecdff5221e83c9946787b8d152c6d83b3a5"} Sep 08 10:19:02.607731 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.607710 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8299s" event={"ID":"cf550904-a7ea-4cd1-8843-766b17279143","Type":"ContainerStarted","Data":"e3fea0847121796a5ec677051a271d66c3bcc8240bf7f819ba3b6d1df8a08d11"} Sep 08 10:19:02.608695 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.608671 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" event={"ID":"3a437240-6ceb-4a5f-b994-4119492c928f","Type":"ContainerStarted","Data":"d14b07073517ae00401420efafb146a33b0e62f39fab04bca8d2dc8930cfd0ef"} Sep 08 10:19:02.614053 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.614031 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:19:02.614447 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.614428 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:19:02.703240 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.703177 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c6cc7594-sqsnv"] Sep 08 10:19:02.856619 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.856589 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 10:19:02.870711 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.870673 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.872714 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.872686 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 08 10:19:02.873018 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.872762 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x7rgr\"" Sep 08 10:19:02.873115 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.873028 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 08 10:19:02.873115 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.873028 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 08 10:19:02.873351 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.873332 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 08 10:19:02.873760 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.873722 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 08 10:19:02.873874 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.873769 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 08 10:19:02.873874 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.873728 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 08 10:19:02.873989 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.873728 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 08 10:19:02.873989 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.873725 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 08 10:19:02.878012 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.877990 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 10:19:02.889670 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.889645 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1437588c-3afd-463c-9217-8cef10e86cca-config-out\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.889781 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.889690 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.889781 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.889730 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1437588c-3afd-463c-9217-8cef10e86cca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.889781 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.889757 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1437588c-3afd-463c-9217-8cef10e86cca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.889944 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.889790 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jthzm\" (UniqueName: \"kubernetes.io/projected/1437588c-3afd-463c-9217-8cef10e86cca-kube-api-access-jthzm\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.889944 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.889830 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-config-volume\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.889944 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.889857 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.890074 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.889963 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.890111 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.890086 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.890156 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.890121 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1437588c-3afd-463c-9217-8cef10e86cca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.890156 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.890148 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.890241 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.890219 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-web-config\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.890295 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.890249 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1437588c-3afd-463c-9217-8cef10e86cca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.990657 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.990628 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1437588c-3afd-463c-9217-8cef10e86cca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.990801 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.990665 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1437588c-3afd-463c-9217-8cef10e86cca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.990801 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.990691 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jthzm\" (UniqueName: \"kubernetes.io/projected/1437588c-3afd-463c-9217-8cef10e86cca-kube-api-access-jthzm\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.990977 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.990955 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-config-volume\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.991030 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.990998 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.991090 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.991074 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.991148 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.991111 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.991148 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.991139 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1437588c-3afd-463c-9217-8cef10e86cca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.991249 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:19:02.991200 2561 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 08 10:19:02.991299 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:19:02.991265 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-secret-alertmanager-main-tls podName:1437588c-3afd-463c-9217-8cef10e86cca nodeName:}" failed. No retries permitted until 2026-09-08 10:19:03.491244069 +0000 UTC m=+119.860509360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "1437588c-3afd-463c-9217-8cef10e86cca") : secret "alertmanager-main-tls" not found Sep 08 10:19:02.991447 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.991419 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1437588c-3afd-463c-9217-8cef10e86cca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.991671 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.991560 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.991671 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.991571 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1437588c-3afd-463c-9217-8cef10e86cca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.991671 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.991604 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-web-config\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.991671 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.991632 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1437588c-3afd-463c-9217-8cef10e86cca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.991671 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.991668 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1437588c-3afd-463c-9217-8cef10e86cca-config-out\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.992200 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.991696 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.992200 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:19:02.991929 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1437588c-3afd-463c-9217-8cef10e86cca-alertmanager-trusted-ca-bundle podName:1437588c-3afd-463c-9217-8cef10e86cca nodeName:}" failed. No retries permitted until 2026-09-08 10:19:03.491910492 +0000 UTC m=+119.861175768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/1437588c-3afd-463c-9217-8cef10e86cca-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "1437588c-3afd-463c-9217-8cef10e86cca") : configmap references non-existent config key: ca-bundle.crt Sep 08 10:19:02.994334 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.994310 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1437588c-3afd-463c-9217-8cef10e86cca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.994489 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.994468 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.994723 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.994700 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.994771 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.994737 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.994846 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.994823 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.994880 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.994868 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-config-volume\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.994948 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.994928 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1437588c-3afd-463c-9217-8cef10e86cca-config-out\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.995904 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.995885 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-web-config\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:02.999147 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:02.999127 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jthzm\" (UniqueName: \"kubernetes.io/projected/1437588c-3afd-463c-9217-8cef10e86cca-kube-api-access-jthzm\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:03.495647 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.495614 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:03.495817 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.495673 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1437588c-3afd-463c-9217-8cef10e86cca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:03.496369 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.496342 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1437588c-3afd-463c-9217-8cef10e86cca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:03.497840 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.497820 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1437588c-3afd-463c-9217-8cef10e86cca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1437588c-3afd-463c-9217-8cef10e86cca\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:03.614123 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.614094 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" event={"ID":"97778083-73ca-43eb-9454-1d3a28e8528a","Type":"ContainerStarted","Data":"b506bf765d99bd9207b932bb1a12636f0ec778e3b7b64e59511be6ccf68aaabf"} Sep 08 10:19:03.784323 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.784274 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 10:19:03.849434 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.849406 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7697b4fdb7-76k8c"] Sep 08 10:19:03.870555 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.869631 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7697b4fdb7-76k8c"] Sep 08 10:19:03.870555 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.869776 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:03.873744 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.873722 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 08 10:19:03.873843 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.873724 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 08 10:19:03.873903 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.873725 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c3o7skmue60v9\"" Sep 08 10:19:03.873903 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.873864 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-zlbwb\"" Sep 08 10:19:03.874052 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.874001 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 08 10:19:03.874052 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.874005 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 08 10:19:03.874277 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.874257 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 08 10:19:03.898062 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.898037 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-thanos-querier-tls\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:03.898164 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.898082 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:03.898164 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.898108 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:03.898247 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.898195 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:03.898247 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.898234 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:03.898319 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.898267 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t8qz\" (UniqueName: \"kubernetes.io/projected/91f35eca-0c71-4f67-b041-3fffa2d7c256-kube-api-access-5t8qz\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:03.898319 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.898306 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-grpc-tls\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:03.898394 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.898332 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91f35eca-0c71-4f67-b041-3fffa2d7c256-metrics-client-ca\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:03.999475 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.999449 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-thanos-querier-tls\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:03.999620 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.999505 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:03.999620 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.999573 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:03.999620 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.999616 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:03.999789 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.999631 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:03.999789 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.999754 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t8qz\" (UniqueName: \"kubernetes.io/projected/91f35eca-0c71-4f67-b041-3fffa2d7c256-kube-api-access-5t8qz\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:03.999903 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.999794 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-grpc-tls\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:03.999903 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:03.999832 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91f35eca-0c71-4f67-b041-3fffa2d7c256-metrics-client-ca\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:04.000615 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:04.000586 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91f35eca-0c71-4f67-b041-3fffa2d7c256-metrics-client-ca\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:04.002242 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:04.002212 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-thanos-querier-tls\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:04.002375 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:04.002354 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:04.002548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:04.002502 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:04.003091 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:04.002981 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:04.003091 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:04.003052 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:04.003091 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:04.003058 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91f35eca-0c71-4f67-b041-3fffa2d7c256-secret-grpc-tls\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:04.010667 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:04.010646 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t8qz\" (UniqueName: \"kubernetes.io/projected/91f35eca-0c71-4f67-b041-3fffa2d7c256-kube-api-access-5t8qz\") pod \"thanos-querier-7697b4fdb7-76k8c\" (UID: \"91f35eca-0c71-4f67-b041-3fffa2d7c256\") " pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:04.096715 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:04.096692 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 10:19:04.104259 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:19:04.104237 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1437588c_3afd_463c_9217_8cef10e86cca.slice/crio-21e4fe45fe02876a7d4ccfd1e25f98a87ea293c916e589b09079bf83a22b59d2 WatchSource:0}: Error finding container 21e4fe45fe02876a7d4ccfd1e25f98a87ea293c916e589b09079bf83a22b59d2: Status 404 returned error can't find the container with id 21e4fe45fe02876a7d4ccfd1e25f98a87ea293c916e589b09079bf83a22b59d2 Sep 08 10:19:04.218035 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:04.218008 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-zlbwb\"" Sep 08 10:19:04.226873 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:04.226851 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:04.511726 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:04.511705 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7697b4fdb7-76k8c"] Sep 08 10:19:04.513411 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:19:04.513385 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91f35eca_0c71_4f67_b041_3fffa2d7c256.slice/crio-1e40aed243f6a12f0c260722331fe2798978a00eacad3f3a5fd89297c9b314ab WatchSource:0}: Error finding container 1e40aed243f6a12f0c260722331fe2798978a00eacad3f3a5fd89297c9b314ab: Status 404 returned error can't find the container with id 1e40aed243f6a12f0c260722331fe2798978a00eacad3f3a5fd89297c9b314ab Sep 08 10:19:04.619222 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:04.619190 2561 generic.go:356] "Generic (PLEG): container finished" podID="cf550904-a7ea-4cd1-8843-766b17279143" containerID="41be079a00fee03aed2160e101c53f3b99c59a85b77f252adcfcf91892e2a375" exitCode=0 Sep 08 10:19:04.619576 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:04.619260 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8299s" event={"ID":"cf550904-a7ea-4cd1-8843-766b17279143","Type":"ContainerDied","Data":"41be079a00fee03aed2160e101c53f3b99c59a85b77f252adcfcf91892e2a375"} Sep 08 10:19:04.620681 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:04.620656 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" event={"ID":"91f35eca-0c71-4f67-b041-3fffa2d7c256","Type":"ContainerStarted","Data":"1e40aed243f6a12f0c260722331fe2798978a00eacad3f3a5fd89297c9b314ab"} Sep 08 10:19:04.621696 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:04.621661 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1437588c-3afd-463c-9217-8cef10e86cca","Type":"ContainerStarted","Data":"21e4fe45fe02876a7d4ccfd1e25f98a87ea293c916e589b09079bf83a22b59d2"} Sep 08 10:19:05.581766 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:05.581737 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8nhg6" Sep 08 10:19:05.631931 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:05.631673 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8299s" event={"ID":"cf550904-a7ea-4cd1-8843-766b17279143","Type":"ContainerStarted","Data":"1065aeb27fca6d25982d543aa79681f410f6f60214b4ca62152d26714c7789b2"} Sep 08 10:19:05.631931 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:05.631740 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8299s" event={"ID":"cf550904-a7ea-4cd1-8843-766b17279143","Type":"ContainerStarted","Data":"628978d59639beae9a6fc7c091aa1280f5de26e126b0833a3b44ca849cb10e73"} Sep 08 10:19:05.654712 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:05.654655 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8299s" podStartSLOduration=2.916879691 podStartE2EDuration="4.65463263s" totalImagesPullingTime="1.653709667s" totalInitContainerRuntime="84.043272ms" isStatefulPod=true podCreationTimestamp="2026-09-08 10:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:19:05.654027717 +0000 UTC m=+122.023293017" watchObservedRunningTime="2026-09-08 10:19:05.65463263 +0000 UTC m=+122.023897928" Sep 08 10:19:06.163083 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.163050 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6fcd85b9b8-q85w4"] Sep 08 10:19:06.188944 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.188919 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6fcd85b9b8-q85w4"] Sep 08 10:19:06.189106 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.189034 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.191760 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.191737 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 08 10:19:06.192463 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.192439 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 08 10:19:06.192612 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.192462 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-n26s9\"" Sep 08 10:19:06.192612 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.192511 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 08 10:19:06.192612 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.192513 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8gf3epo8qf84g\"" Sep 08 10:19:06.192782 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.192674 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 08 10:19:06.219858 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.219833 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-metrics-server-audit-profiles\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.219980 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.219892 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-secret-metrics-server-client-certs\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.219980 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.219922 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxd9r\" (UniqueName: \"kubernetes.io/projected/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-kube-api-access-dxd9r\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.220074 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.219995 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.220074 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.220045 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-secret-metrics-server-tls\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.220074 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.220062 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-client-ca-bundle\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.220193 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.220119 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-audit-log\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.320940 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.320913 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxd9r\" (UniqueName: \"kubernetes.io/projected/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-kube-api-access-dxd9r\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.321087 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.320982 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.321087 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.321028 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-secret-metrics-server-tls\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.321087 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.321051 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-client-ca-bundle\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.321243 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.321090 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-audit-log\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.321243 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.321126 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-metrics-server-audit-profiles\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.321243 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.321224 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-secret-metrics-server-client-certs\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.321590 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.321560 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-audit-log\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.321981 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.321952 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.322511 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.322461 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-metrics-server-audit-profiles\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.323950 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.323928 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-client-ca-bundle\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.324054 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.323953 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-secret-metrics-server-client-certs\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.324319 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.324302 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-secret-metrics-server-tls\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.330042 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.330013 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxd9r\" (UniqueName: \"kubernetes.io/projected/01e8dec6-55f6-4a91-bff3-a8ccf61e19bc-kube-api-access-dxd9r\") pod \"metrics-server-6fcd85b9b8-q85w4\" (UID: \"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc\") " pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:06.499023 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:06.498985 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:07.293639 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.292831 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-695f789945-fz72g"] Sep 08 10:19:07.314428 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.314099 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-695f789945-fz72g"] Sep 08 10:19:07.314428 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.314223 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.332289 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.331827 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-serving-cert\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.332289 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.331936 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-config\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.332289 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.331973 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-oauth-config\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.332289 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.332005 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-service-ca\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.332289 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.332040 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-trusted-ca-bundle\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.332289 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.332094 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz6mf\" (UniqueName: \"kubernetes.io/projected/43038ed5-4a97-46d3-aaa1-c43a61327e22-kube-api-access-lz6mf\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.332289 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.332123 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-oauth-serving-cert\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.395916 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.395840 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6fcd85b9b8-q85w4"] Sep 08 10:19:07.401768 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:19:07.401644 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e8dec6_55f6_4a91_bff3_a8ccf61e19bc.slice/crio-1f9748d08bc74a5cf7eea28d1990cf7d721caf1cc054c8ec1ed6f2aee6eacc1a WatchSource:0}: Error finding container 1f9748d08bc74a5cf7eea28d1990cf7d721caf1cc054c8ec1ed6f2aee6eacc1a: Status 404 returned error can't find the container with id 1f9748d08bc74a5cf7eea28d1990cf7d721caf1cc054c8ec1ed6f2aee6eacc1a Sep 08 10:19:07.433565 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.432947 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-config\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.433565 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.432992 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-oauth-config\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.433565 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.433017 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-service-ca\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.433565 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.433049 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-trusted-ca-bundle\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.433565 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.433077 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz6mf\" (UniqueName: \"kubernetes.io/projected/43038ed5-4a97-46d3-aaa1-c43a61327e22-kube-api-access-lz6mf\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.433565 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.433106 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-oauth-serving-cert\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.433565 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.433182 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-serving-cert\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.433992 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.433966 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-service-ca\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.434050 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.433974 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-oauth-serving-cert\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.434616 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.434550 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-trusted-ca-bundle\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.434976 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.434953 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-config\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.437681 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.437654 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-oauth-config\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.438408 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.438355 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-serving-cert\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.444340 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.444300 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz6mf\" (UniqueName: \"kubernetes.io/projected/43038ed5-4a97-46d3-aaa1-c43a61327e22-kube-api-access-lz6mf\") pod \"console-695f789945-fz72g\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.643053 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.642573 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" event={"ID":"3a437240-6ceb-4a5f-b994-4119492c928f","Type":"ContainerStarted","Data":"83357d85d9cfd92f15cd9a5d0265f9237afb5cfbad70f5eb4a8aaaaf7f943e92"} Sep 08 10:19:07.643053 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.642634 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" event={"ID":"3a437240-6ceb-4a5f-b994-4119492c928f","Type":"ContainerStarted","Data":"f31426ce50107bdf204f8e9dbd42ff4eeb5fe817a3f13be011cae5455c73db9b"} Sep 08 10:19:07.643053 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.642654 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" event={"ID":"3a437240-6ceb-4a5f-b994-4119492c928f","Type":"ContainerStarted","Data":"d5796be25eafe10b7991d9f86c917f23def9c87655090831cfb60b1686337c63"} Sep 08 10:19:07.643860 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.643812 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" event={"ID":"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc","Type":"ContainerStarted","Data":"1f9748d08bc74a5cf7eea28d1990cf7d721caf1cc054c8ec1ed6f2aee6eacc1a"} Sep 08 10:19:07.645472 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.645426 2561 generic.go:356] "Generic (PLEG): container finished" podID="1437588c-3afd-463c-9217-8cef10e86cca" containerID="f707d735d71184f196b92c483b7cc9843e6579e001be6ec1e8e936209fdf852f" exitCode=0 Sep 08 10:19:07.645636 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.645478 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1437588c-3afd-463c-9217-8cef10e86cca","Type":"ContainerDied","Data":"f707d735d71184f196b92c483b7cc9843e6579e001be6ec1e8e936209fdf852f"} Sep 08 10:19:07.648063 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.648041 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" event={"ID":"91f35eca-0c71-4f67-b041-3fffa2d7c256","Type":"ContainerStarted","Data":"85d2c15eda5896f3dc7eb700ffc268a5d62535b0c4c2e21f117adc428cf8981f"} Sep 08 10:19:07.648164 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.648116 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" event={"ID":"91f35eca-0c71-4f67-b041-3fffa2d7c256","Type":"ContainerStarted","Data":"cb5a974927f7015ca066806433965643b0e0f4c19c108dddbd3a9b11ecf7d419"} Sep 08 10:19:07.648222 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.648159 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" event={"ID":"91f35eca-0c71-4f67-b041-3fffa2d7c256","Type":"ContainerStarted","Data":"d3ffddd51c9eac19133e2b402a4e79e1275664207fa86e28a84c4b8e12a8efba"} Sep 08 10:19:07.666635 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.666566 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-hjzj2" podStartSLOduration=1.9549646 podStartE2EDuration="6.666547786s" totalImagesPullingTime="4.711583186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:19:07.666250719 +0000 UTC m=+124.035516017" watchObservedRunningTime="2026-09-08 10:19:07.666547786 +0000 UTC m=+124.035813080" Sep 08 10:19:07.667159 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.667136 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:07.728580 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:07.728459 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:19:08.157368 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:08.157339 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-695f789945-fz72g"] Sep 08 10:19:08.213199 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:19:08.213163 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43038ed5_4a97_46d3_aaa1_c43a61327e22.slice/crio-12accf1e3622091a808b1de684e4a125986d1747990d05fd28801e3cacfd33ef WatchSource:0}: Error finding container 12accf1e3622091a808b1de684e4a125986d1747990d05fd28801e3cacfd33ef: Status 404 returned error can't find the container with id 12accf1e3622091a808b1de684e4a125986d1747990d05fd28801e3cacfd33ef Sep 08 10:19:08.656757 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:08.656648 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695f789945-fz72g" event={"ID":"43038ed5-4a97-46d3-aaa1-c43a61327e22","Type":"ContainerStarted","Data":"b26c3999efb29a905547eb95c1244ebc9f1d840702cc99505b2a2036f585eca7"} Sep 08 10:19:08.656757 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:08.656702 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695f789945-fz72g" event={"ID":"43038ed5-4a97-46d3-aaa1-c43a61327e22","Type":"ContainerStarted","Data":"12accf1e3622091a808b1de684e4a125986d1747990d05fd28801e3cacfd33ef"} Sep 08 10:19:08.664050 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:08.663760 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" event={"ID":"97778083-73ca-43eb-9454-1d3a28e8528a","Type":"ContainerStarted","Data":"6092b858c21669414ada75c0376997cb9fa74bf1db0007dff21a3ec187bf9822"} Sep 08 10:19:08.683093 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:08.681678 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-695f789945-fz72g" podStartSLOduration=1.681659168 podStartE2EDuration="1.681659168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:19:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:19:08.680643936 +0000 UTC m=+125.049909245" watchObservedRunningTime="2026-09-08 10:19:08.681659168 +0000 UTC m=+125.050924466" Sep 08 10:19:08.701915 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:08.701474 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-t7bpc" podStartSLOduration=2.099342271 podStartE2EDuration="7.70145477s" totalImagesPullingTime="5.602112499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:19:08.699140721 +0000 UTC m=+125.068406019" watchObservedRunningTime="2026-09-08 10:19:08.70145477 +0000 UTC m=+125.070720065" Sep 08 10:19:10.674707 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:10.674670 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1437588c-3afd-463c-9217-8cef10e86cca","Type":"ContainerStarted","Data":"03f1b4e0bcab6b6f157b03516b59962f9c3ef7eb0ab7eb829d2a90ed72da6a9d"} Sep 08 10:19:10.677293 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:10.677269 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" event={"ID":"91f35eca-0c71-4f67-b041-3fffa2d7c256","Type":"ContainerStarted","Data":"4d15e2134292e6b35925971419477ddf0f554c013edde88a913375262ea17c4f"} Sep 08 10:19:10.679133 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:10.679105 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" event={"ID":"01e8dec6-55f6-4a91-bff3-a8ccf61e19bc","Type":"ContainerStarted","Data":"e6d82f31ff335bdd8934326afde6fd70ecebefdaace0c3eef9bbb89a5d4920ac"} Sep 08 10:19:10.702644 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:10.702599 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" podStartSLOduration=1.6188892369999999 podStartE2EDuration="4.702578092s" totalImagesPullingTime="3.083688855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:19:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:19:10.700854086 +0000 UTC m=+127.070119383" watchObservedRunningTime="2026-09-08 10:19:10.702578092 +0000 UTC m=+127.071843387" Sep 08 10:19:11.686696 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:11.686656 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" event={"ID":"91f35eca-0c71-4f67-b041-3fffa2d7c256","Type":"ContainerStarted","Data":"1cebdcd509da479a49b2d64b0acb8f32d74a87c239e6cff202e7938b49bf3529"} Sep 08 10:19:11.687231 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:11.686780 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:11.687231 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:11.686837 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" event={"ID":"91f35eca-0c71-4f67-b041-3fffa2d7c256","Type":"ContainerStarted","Data":"e9580418ebeb4e1482fc999aa02df08f5cf00b47a998579d89292283b58557ea"} Sep 08 10:19:11.689919 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:11.689895 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1437588c-3afd-463c-9217-8cef10e86cca","Type":"ContainerStarted","Data":"8276ee111116ecd92519fe229baba10f39df276c70811f3307b7af062567939b"} Sep 08 10:19:11.690023 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:11.689948 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1437588c-3afd-463c-9217-8cef10e86cca","Type":"ContainerStarted","Data":"4ad7d9777a8e58b21a0f32f7fe12498e5fe2ce4e61a4e3d5b0971f489d55f415"} Sep 08 10:19:11.690023 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:11.689975 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1437588c-3afd-463c-9217-8cef10e86cca","Type":"ContainerStarted","Data":"a9d9e15e586e652ffef5a07a92c0a6be247f21032e9d311b33be38e15543a723"} Sep 08 10:19:11.690023 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:11.690000 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1437588c-3afd-463c-9217-8cef10e86cca","Type":"ContainerStarted","Data":"117d6816ecc51208373dfd078919dc8344c336354921c45fde08d5711a9da01b"} Sep 08 10:19:11.690121 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:11.690025 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1437588c-3afd-463c-9217-8cef10e86cca","Type":"ContainerStarted","Data":"281225e7dde2254a14100e27333c9525c986f1460639f5eeb6f34c3162925c8a"} Sep 08 10:19:11.712208 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:11.712161 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" podStartSLOduration=3.183228363 podStartE2EDuration="8.712145774s" totalImagesPullingTime="5.528917411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:19:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:19:11.710009877 +0000 UTC m=+128.079275187" watchObservedRunningTime="2026-09-08 10:19:11.712145774 +0000 UTC m=+128.081411071" Sep 08 10:19:11.739907 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:11.739869 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.856487494 podStartE2EDuration="9.739851873s" totalImagesPullingTime="5.863978061s" totalInitContainerRuntime="19.386318ms" isStatefulPod=false podCreationTimestamp="2026-09-08 10:19:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:19:11.737640666 +0000 UTC m=+128.106905963" watchObservedRunningTime="2026-09-08 10:19:11.739851873 +0000 UTC m=+128.109117170" Sep 08 10:19:12.699610 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:12.699582 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7697b4fdb7-76k8c" Sep 08 10:19:12.980830 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:12.980286 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs\") pod \"network-metrics-daemon-pr8g9\" (UID: \"db4c4c5b-19b6-439a-bcd2-0631f7626479\") " pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:19:12.983043 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:12.983012 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4c4c5b-19b6-439a-bcd2-0631f7626479-metrics-certs\") pod \"network-metrics-daemon-pr8g9\" (UID: \"db4c4c5b-19b6-439a-bcd2-0631f7626479\") " pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:19:13.155902 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:13.155870 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ljzsh\"" Sep 08 10:19:13.164785 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:13.164754 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8g9" Sep 08 10:19:13.473495 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:13.473441 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pr8g9"] Sep 08 10:19:13.475648 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:19:13.475616 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb4c4c5b_19b6_439a_bcd2_0631f7626479.slice/crio-bba73981f5a258ba16f99f0233a767e86d060d7db78b4bad201e39059c8b1675 WatchSource:0}: Error finding container bba73981f5a258ba16f99f0233a767e86d060d7db78b4bad201e39059c8b1675: Status 404 returned error can't find the container with id bba73981f5a258ba16f99f0233a767e86d060d7db78b4bad201e39059c8b1675 Sep 08 10:19:13.699101 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:13.699027 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pr8g9" event={"ID":"db4c4c5b-19b6-439a-bcd2-0631f7626479","Type":"ContainerStarted","Data":"bba73981f5a258ba16f99f0233a767e86d060d7db78b4bad201e39059c8b1675"} Sep 08 10:19:15.710614 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:15.710578 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pr8g9" event={"ID":"db4c4c5b-19b6-439a-bcd2-0631f7626479","Type":"ContainerStarted","Data":"f7fa89ad17355b2975879d81fbeb76e461f5900982962a8fd4cf72bfc822f6df"} Sep 08 10:19:15.711027 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:15.710629 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pr8g9" event={"ID":"db4c4c5b-19b6-439a-bcd2-0631f7626479","Type":"ContainerStarted","Data":"ef2ebbe841a5d94292b1d7c730a21d1eea0278bd2c81da4ce66a50e8507016fa"} Sep 08 10:19:15.734612 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:15.734566 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pr8g9" podStartSLOduration=128.998040628 podStartE2EDuration="2m10.734552013s" totalImagesPullingTime="1.736511385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:19:15.732932087 +0000 UTC m=+132.102197384" watchObservedRunningTime="2026-09-08 10:19:15.734552013 +0000 UTC m=+132.103817310" Sep 08 10:19:17.668035 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:17.668004 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:17.668469 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:17.668059 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:17.672616 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:17.672595 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:17.722211 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:17.722187 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-695f789945-fz72g" Sep 08 10:19:17.780511 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:17.780489 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-547759fdf4-7blpv"] Sep 08 10:19:22.748328 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:22.748265 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-59557ddf67-5k75j" podUID="2dd92822-b06b-4eaa-8cf7-cc210c5de102" containerName="registry" containerID="cri-o://6e4edac8971bc3d8c8b9b5aec7b8a9890b1b3b0be7683d834e95666894458dfa" gracePeriod=30 Sep 08 10:19:22.996574 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:22.996549 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:19:23.055312 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.055250 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls\") pod \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " Sep 08 10:19:23.055312 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.055289 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prr8d\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-kube-api-access-prr8d\") pod \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " Sep 08 10:19:23.055473 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.055349 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2dd92822-b06b-4eaa-8cf7-cc210c5de102-image-registry-private-configuration\") pod \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " Sep 08 10:19:23.055473 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.055381 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2dd92822-b06b-4eaa-8cf7-cc210c5de102-ca-trust-extracted\") pod \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " Sep 08 10:19:23.055473 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.055461 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2dd92822-b06b-4eaa-8cf7-cc210c5de102-installation-pull-secrets\") pod \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " Sep 08 10:19:23.055658 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.055498 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2dd92822-b06b-4eaa-8cf7-cc210c5de102-trusted-ca\") pod \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " Sep 08 10:19:23.055658 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.055552 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-certificates\") pod \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " Sep 08 10:19:23.055658 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.055601 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-bound-sa-token\") pod \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\" (UID: \"2dd92822-b06b-4eaa-8cf7-cc210c5de102\") " Sep 08 10:19:23.055959 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.055927 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dd92822-b06b-4eaa-8cf7-cc210c5de102-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2dd92822-b06b-4eaa-8cf7-cc210c5de102" (UID: "2dd92822-b06b-4eaa-8cf7-cc210c5de102"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:19:23.055959 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.055943 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2dd92822-b06b-4eaa-8cf7-cc210c5de102" (UID: "2dd92822-b06b-4eaa-8cf7-cc210c5de102"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:19:23.057804 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.057774 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dd92822-b06b-4eaa-8cf7-cc210c5de102-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2dd92822-b06b-4eaa-8cf7-cc210c5de102" (UID: "2dd92822-b06b-4eaa-8cf7-cc210c5de102"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:19:23.057932 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.057908 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2dd92822-b06b-4eaa-8cf7-cc210c5de102" (UID: "2dd92822-b06b-4eaa-8cf7-cc210c5de102"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:19:23.058127 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.058110 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-kube-api-access-prr8d" (OuterVolumeSpecName: "kube-api-access-prr8d") pod "2dd92822-b06b-4eaa-8cf7-cc210c5de102" (UID: "2dd92822-b06b-4eaa-8cf7-cc210c5de102"). InnerVolumeSpecName "kube-api-access-prr8d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:19:23.058127 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.058115 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2dd92822-b06b-4eaa-8cf7-cc210c5de102" (UID: "2dd92822-b06b-4eaa-8cf7-cc210c5de102"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:19:23.058394 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.058375 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dd92822-b06b-4eaa-8cf7-cc210c5de102-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2dd92822-b06b-4eaa-8cf7-cc210c5de102" (UID: "2dd92822-b06b-4eaa-8cf7-cc210c5de102"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:19:23.063795 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.063771 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd92822-b06b-4eaa-8cf7-cc210c5de102-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2dd92822-b06b-4eaa-8cf7-cc210c5de102" (UID: "2dd92822-b06b-4eaa-8cf7-cc210c5de102"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:19:23.156262 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.156240 2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-bound-sa-token\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:23.156262 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.156259 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:23.156386 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.156268 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prr8d\" (UniqueName: \"kubernetes.io/projected/2dd92822-b06b-4eaa-8cf7-cc210c5de102-kube-api-access-prr8d\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:23.156386 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.156278 2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2dd92822-b06b-4eaa-8cf7-cc210c5de102-image-registry-private-configuration\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:23.156386 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.156287 2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2dd92822-b06b-4eaa-8cf7-cc210c5de102-ca-trust-extracted\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:23.156386 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.156295 2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2dd92822-b06b-4eaa-8cf7-cc210c5de102-installation-pull-secrets\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:23.156386 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.156303 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2dd92822-b06b-4eaa-8cf7-cc210c5de102-trusted-ca\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:23.156386 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.156310 2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2dd92822-b06b-4eaa-8cf7-cc210c5de102-registry-certificates\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:23.742685 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.742656 2561 generic.go:356] "Generic (PLEG): container finished" podID="2dd92822-b06b-4eaa-8cf7-cc210c5de102" containerID="6e4edac8971bc3d8c8b9b5aec7b8a9890b1b3b0be7683d834e95666894458dfa" exitCode=0 Sep 08 10:19:23.742839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.742706 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59557ddf67-5k75j" Sep 08 10:19:23.742839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.742748 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59557ddf67-5k75j" event={"ID":"2dd92822-b06b-4eaa-8cf7-cc210c5de102","Type":"ContainerDied","Data":"6e4edac8971bc3d8c8b9b5aec7b8a9890b1b3b0be7683d834e95666894458dfa"} Sep 08 10:19:23.742839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.742801 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59557ddf67-5k75j" event={"ID":"2dd92822-b06b-4eaa-8cf7-cc210c5de102","Type":"ContainerDied","Data":"519bda657f76dda313289cadacbcd0f88ce70821d673eea8d49e8e329d8792d6"} Sep 08 10:19:23.742839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.742831 2561 scope.go:117] "RemoveContainer" containerID="6e4edac8971bc3d8c8b9b5aec7b8a9890b1b3b0be7683d834e95666894458dfa" Sep 08 10:19:23.755797 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.755651 2561 scope.go:117] "RemoveContainer" containerID="6e4edac8971bc3d8c8b9b5aec7b8a9890b1b3b0be7683d834e95666894458dfa" Sep 08 10:19:23.755994 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:19:23.755910 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e4edac8971bc3d8c8b9b5aec7b8a9890b1b3b0be7683d834e95666894458dfa\": container with ID starting with 6e4edac8971bc3d8c8b9b5aec7b8a9890b1b3b0be7683d834e95666894458dfa not found: ID does not exist" containerID="6e4edac8971bc3d8c8b9b5aec7b8a9890b1b3b0be7683d834e95666894458dfa" Sep 08 10:19:23.755994 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.755932 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e4edac8971bc3d8c8b9b5aec7b8a9890b1b3b0be7683d834e95666894458dfa"} err="failed to get container status \"6e4edac8971bc3d8c8b9b5aec7b8a9890b1b3b0be7683d834e95666894458dfa\": rpc error: code = NotFound desc = could not find container \"6e4edac8971bc3d8c8b9b5aec7b8a9890b1b3b0be7683d834e95666894458dfa\": container with ID starting with 6e4edac8971bc3d8c8b9b5aec7b8a9890b1b3b0be7683d834e95666894458dfa not found: ID does not exist" Sep 08 10:19:23.767036 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.767015 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59557ddf67-5k75j"] Sep 08 10:19:23.771093 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:23.771075 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-59557ddf67-5k75j"] Sep 08 10:19:24.144958 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:24.144894 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dd92822-b06b-4eaa-8cf7-cc210c5de102" path="/var/lib/kubelet/pods/2dd92822-b06b-4eaa-8cf7-cc210c5de102/volumes" Sep 08 10:19:26.499498 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:26.499467 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:26.499884 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:26.499519 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:29.640487 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:29.640451 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c6cc7594-sqsnv" podUID="7156d04a-f799-4bea-a216-6a2a846cef60" containerName="console" containerID="cri-o://75a6cb5ee591ec57a632194d9c7dbd09489ae86c79ecf57eee7d59ab93c9ce23" gracePeriod=15 Sep 08 10:19:29.767262 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:29.767232 2561 generic.go:356] "Generic (PLEG): container finished" podID="7156d04a-f799-4bea-a216-6a2a846cef60" containerID="75a6cb5ee591ec57a632194d9c7dbd09489ae86c79ecf57eee7d59ab93c9ce23" exitCode=0 Sep 08 10:19:29.767362 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:29.767305 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6cc7594-sqsnv" event={"ID":"7156d04a-f799-4bea-a216-6a2a846cef60","Type":"ContainerDied","Data":"75a6cb5ee591ec57a632194d9c7dbd09489ae86c79ecf57eee7d59ab93c9ce23"} Sep 08 10:19:29.928226 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:29.928173 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:19:30.005716 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.005686 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7156d04a-f799-4bea-a216-6a2a846cef60-console-serving-cert\") pod \"7156d04a-f799-4bea-a216-6a2a846cef60\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " Sep 08 10:19:30.005847 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.005725 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-oauth-serving-cert\") pod \"7156d04a-f799-4bea-a216-6a2a846cef60\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " Sep 08 10:19:30.005847 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.005746 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-service-ca\") pod \"7156d04a-f799-4bea-a216-6a2a846cef60\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " Sep 08 10:19:30.005847 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.005768 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-console-config\") pod \"7156d04a-f799-4bea-a216-6a2a846cef60\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " Sep 08 10:19:30.005847 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.005816 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7156d04a-f799-4bea-a216-6a2a846cef60-console-oauth-config\") pod \"7156d04a-f799-4bea-a216-6a2a846cef60\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " Sep 08 10:19:30.005847 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.005843 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx9nw\" (UniqueName: \"kubernetes.io/projected/7156d04a-f799-4bea-a216-6a2a846cef60-kube-api-access-rx9nw\") pod \"7156d04a-f799-4bea-a216-6a2a846cef60\" (UID: \"7156d04a-f799-4bea-a216-6a2a846cef60\") " Sep 08 10:19:30.006109 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.006083 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7156d04a-f799-4bea-a216-6a2a846cef60" (UID: "7156d04a-f799-4bea-a216-6a2a846cef60"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:19:30.006151 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.006108 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-service-ca" (OuterVolumeSpecName: "service-ca") pod "7156d04a-f799-4bea-a216-6a2a846cef60" (UID: "7156d04a-f799-4bea-a216-6a2a846cef60"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:19:30.006198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.006174 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-console-config" (OuterVolumeSpecName: "console-config") pod "7156d04a-f799-4bea-a216-6a2a846cef60" (UID: "7156d04a-f799-4bea-a216-6a2a846cef60"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:19:30.007934 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.007908 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7156d04a-f799-4bea-a216-6a2a846cef60-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7156d04a-f799-4bea-a216-6a2a846cef60" (UID: "7156d04a-f799-4bea-a216-6a2a846cef60"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:19:30.007934 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.007926 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7156d04a-f799-4bea-a216-6a2a846cef60-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7156d04a-f799-4bea-a216-6a2a846cef60" (UID: "7156d04a-f799-4bea-a216-6a2a846cef60"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:19:30.008063 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.007964 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7156d04a-f799-4bea-a216-6a2a846cef60-kube-api-access-rx9nw" (OuterVolumeSpecName: "kube-api-access-rx9nw") pod "7156d04a-f799-4bea-a216-6a2a846cef60" (UID: "7156d04a-f799-4bea-a216-6a2a846cef60"). InnerVolumeSpecName "kube-api-access-rx9nw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:19:30.107183 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.107161 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7156d04a-f799-4bea-a216-6a2a846cef60-console-oauth-config\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:30.107183 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.107182 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx9nw\" (UniqueName: \"kubernetes.io/projected/7156d04a-f799-4bea-a216-6a2a846cef60-kube-api-access-rx9nw\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:30.107300 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.107191 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7156d04a-f799-4bea-a216-6a2a846cef60-console-serving-cert\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:30.107300 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.107201 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-oauth-serving-cert\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:30.107300 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.107209 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-service-ca\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:30.107300 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.107218 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7156d04a-f799-4bea-a216-6a2a846cef60-console-config\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:30.772743 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.772720 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6cc7594-sqsnv" Sep 08 10:19:30.773127 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.772720 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6cc7594-sqsnv" event={"ID":"7156d04a-f799-4bea-a216-6a2a846cef60","Type":"ContainerDied","Data":"53f32f018c4715eafb1e55b137564b8433fe90f3d7a6f2510b650c0420fced65"} Sep 08 10:19:30.773127 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.772842 2561 scope.go:117] "RemoveContainer" containerID="75a6cb5ee591ec57a632194d9c7dbd09489ae86c79ecf57eee7d59ab93c9ce23" Sep 08 10:19:30.790383 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.790359 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c6cc7594-sqsnv"] Sep 08 10:19:30.795045 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:30.795025 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c6cc7594-sqsnv"] Sep 08 10:19:32.144456 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:32.144425 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7156d04a-f799-4bea-a216-6a2a846cef60" path="/var/lib/kubelet/pods/7156d04a-f799-4bea-a216-6a2a846cef60/volumes" Sep 08 10:19:42.799507 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:42.799449 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-547759fdf4-7blpv" podUID="dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa" containerName="console" containerID="cri-o://d8d05118de3bf4adde8957a8d2d0fe1fa577b9d796baa1d3592f59b5d358ea59" gracePeriod=15 Sep 08 10:19:43.056788 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.056729 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:19:43.202135 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.202110 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-oauth-serving-cert\") pod \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " Sep 08 10:19:43.202272 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.202140 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-oauth-config\") pod \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " Sep 08 10:19:43.202272 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.202194 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-service-ca\") pod \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " Sep 08 10:19:43.202272 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.202221 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-serving-cert\") pod \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " Sep 08 10:19:43.202272 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.202238 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72mp6\" (UniqueName: \"kubernetes.io/projected/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-kube-api-access-72mp6\") pod \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " Sep 08 10:19:43.202272 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.202254 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-trusted-ca-bundle\") pod \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " Sep 08 10:19:43.202524 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.202276 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-config\") pod \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\" (UID: \"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa\") " Sep 08 10:19:43.202665 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.202633 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa" (UID: "dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:19:43.202728 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.202677 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-service-ca" (OuterVolumeSpecName: "service-ca") pod "dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa" (UID: "dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:19:43.202788 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.202768 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-config" (OuterVolumeSpecName: "console-config") pod "dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa" (UID: "dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:19:43.202831 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.202765 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa" (UID: "dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:19:43.204283 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.204264 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa" (UID: "dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:19:43.204793 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.204761 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-kube-api-access-72mp6" (OuterVolumeSpecName: "kube-api-access-72mp6") pod "dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa" (UID: "dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa"). InnerVolumeSpecName "kube-api-access-72mp6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:19:43.204793 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.204772 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa" (UID: "dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:19:43.303368 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.303340 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-service-ca\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:43.303368 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.303365 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-serving-cert\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:43.303511 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.303375 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72mp6\" (UniqueName: \"kubernetes.io/projected/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-kube-api-access-72mp6\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:43.303511 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.303384 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-trusted-ca-bundle\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:43.303511 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.303393 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-config\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:43.303511 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.303401 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-oauth-serving-cert\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:43.303511 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.303408 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa-console-oauth-config\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:19:43.825868 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.825838 2561 generic.go:356] "Generic (PLEG): container finished" podID="dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa" containerID="d8d05118de3bf4adde8957a8d2d0fe1fa577b9d796baa1d3592f59b5d358ea59" exitCode=0 Sep 08 10:19:43.826218 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.825884 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-547759fdf4-7blpv" Sep 08 10:19:43.826218 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.825925 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-547759fdf4-7blpv" event={"ID":"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa","Type":"ContainerDied","Data":"d8d05118de3bf4adde8957a8d2d0fe1fa577b9d796baa1d3592f59b5d358ea59"} Sep 08 10:19:43.826218 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.825978 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-547759fdf4-7blpv" event={"ID":"dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa","Type":"ContainerDied","Data":"3482d7be8c2138ac9fcdfc401cba93be2dabba84242983768da2f694b8824743"} Sep 08 10:19:43.826218 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.826004 2561 scope.go:117] "RemoveContainer" containerID="d8d05118de3bf4adde8957a8d2d0fe1fa577b9d796baa1d3592f59b5d358ea59" Sep 08 10:19:43.834332 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.834314 2561 scope.go:117] "RemoveContainer" containerID="d8d05118de3bf4adde8957a8d2d0fe1fa577b9d796baa1d3592f59b5d358ea59" Sep 08 10:19:43.834649 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:19:43.834628 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8d05118de3bf4adde8957a8d2d0fe1fa577b9d796baa1d3592f59b5d358ea59\": container with ID starting with d8d05118de3bf4adde8957a8d2d0fe1fa577b9d796baa1d3592f59b5d358ea59 not found: ID does not exist" containerID="d8d05118de3bf4adde8957a8d2d0fe1fa577b9d796baa1d3592f59b5d358ea59" Sep 08 10:19:43.834727 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.834655 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d05118de3bf4adde8957a8d2d0fe1fa577b9d796baa1d3592f59b5d358ea59"} err="failed to get container status \"d8d05118de3bf4adde8957a8d2d0fe1fa577b9d796baa1d3592f59b5d358ea59\": rpc error: code = NotFound desc = could not find container \"d8d05118de3bf4adde8957a8d2d0fe1fa577b9d796baa1d3592f59b5d358ea59\": container with ID starting with d8d05118de3bf4adde8957a8d2d0fe1fa577b9d796baa1d3592f59b5d358ea59 not found: ID does not exist" Sep 08 10:19:43.846686 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.846665 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-547759fdf4-7blpv"] Sep 08 10:19:43.851482 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:43.851455 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-547759fdf4-7blpv"] Sep 08 10:19:44.144485 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:44.144411 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa" path="/var/lib/kubelet/pods/dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa/volumes" Sep 08 10:19:46.504838 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:46.504813 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:19:46.508428 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:19:46.508404 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6fcd85b9b8-q85w4" Sep 08 10:20:26.167596 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.167517 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-54bf4db66-fst9v"] Sep 08 10:20:26.168607 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.168583 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7156d04a-f799-4bea-a216-6a2a846cef60" containerName="console" Sep 08 10:20:26.168607 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.168605 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7156d04a-f799-4bea-a216-6a2a846cef60" containerName="console" Sep 08 10:20:26.168607 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.168615 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa" containerName="console" Sep 08 10:20:26.168778 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.168623 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa" containerName="console" Sep 08 10:20:26.168778 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.168639 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dd92822-b06b-4eaa-8cf7-cc210c5de102" containerName="registry" Sep 08 10:20:26.168778 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.168645 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dd92822-b06b-4eaa-8cf7-cc210c5de102" containerName="registry" Sep 08 10:20:26.168778 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.168714 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc1f1e0d-5b89-4e05-aaf0-549dd3b604fa" containerName="console" Sep 08 10:20:26.168778 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.168725 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dd92822-b06b-4eaa-8cf7-cc210c5de102" containerName="registry" Sep 08 10:20:26.168778 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.168732 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7156d04a-f799-4bea-a216-6a2a846cef60" containerName="console" Sep 08 10:20:26.171770 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.171755 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.174607 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.174567 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 08 10:20:26.174741 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.174711 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-bvzkq\"" Sep 08 10:20:26.175285 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.175268 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 08 10:20:26.175554 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.175520 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 08 10:20:26.175642 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.175623 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 08 10:20:26.176739 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.176716 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 08 10:20:26.180249 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.180232 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 08 10:20:26.184553 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.184490 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-54bf4db66-fst9v"] Sep 08 10:20:26.226357 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.226331 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f0e73e10-7e61-40b7-935e-bedd4b1e481e-secret-telemeter-client\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.226483 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.226361 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0e73e10-7e61-40b7-935e-bedd4b1e481e-serving-certs-ca-bundle\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.226483 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.226392 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f0e73e10-7e61-40b7-935e-bedd4b1e481e-telemeter-client-tls\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.226483 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.226407 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f0e73e10-7e61-40b7-935e-bedd4b1e481e-federate-client-tls\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.226647 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.226474 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx5sv\" (UniqueName: \"kubernetes.io/projected/f0e73e10-7e61-40b7-935e-bedd4b1e481e-kube-api-access-fx5sv\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.226647 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.226511 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0e73e10-7e61-40b7-935e-bedd4b1e481e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.226647 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.226611 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f0e73e10-7e61-40b7-935e-bedd4b1e481e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.226756 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.226661 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0e73e10-7e61-40b7-935e-bedd4b1e481e-metrics-client-ca\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.327258 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.327236 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0e73e10-7e61-40b7-935e-bedd4b1e481e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.327384 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.327268 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f0e73e10-7e61-40b7-935e-bedd4b1e481e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.327384 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.327291 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0e73e10-7e61-40b7-935e-bedd4b1e481e-metrics-client-ca\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.327384 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.327329 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f0e73e10-7e61-40b7-935e-bedd4b1e481e-secret-telemeter-client\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.327384 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.327355 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0e73e10-7e61-40b7-935e-bedd4b1e481e-serving-certs-ca-bundle\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.327633 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.327409 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f0e73e10-7e61-40b7-935e-bedd4b1e481e-telemeter-client-tls\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.327633 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.327513 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f0e73e10-7e61-40b7-935e-bedd4b1e481e-federate-client-tls\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.327633 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.327584 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx5sv\" (UniqueName: \"kubernetes.io/projected/f0e73e10-7e61-40b7-935e-bedd4b1e481e-kube-api-access-fx5sv\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.328180 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.328124 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0e73e10-7e61-40b7-935e-bedd4b1e481e-metrics-client-ca\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.328180 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.328131 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0e73e10-7e61-40b7-935e-bedd4b1e481e-serving-certs-ca-bundle\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.328410 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.328392 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f0e73e10-7e61-40b7-935e-bedd4b1e481e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.329966 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.329946 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f0e73e10-7e61-40b7-935e-bedd4b1e481e-secret-telemeter-client\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.330077 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.330054 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f0e73e10-7e61-40b7-935e-bedd4b1e481e-federate-client-tls\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.330262 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.330241 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f0e73e10-7e61-40b7-935e-bedd4b1e481e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.330302 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.330262 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f0e73e10-7e61-40b7-935e-bedd4b1e481e-telemeter-client-tls\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.336904 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.336884 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx5sv\" (UniqueName: \"kubernetes.io/projected/f0e73e10-7e61-40b7-935e-bedd4b1e481e-kube-api-access-fx5sv\") pod \"telemeter-client-54bf4db66-fst9v\" (UID: \"f0e73e10-7e61-40b7-935e-bedd4b1e481e\") " pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.481509 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.481454 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" Sep 08 10:20:26.763230 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.763206 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-54bf4db66-fst9v"] Sep 08 10:20:26.764959 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:20:26.764934 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e73e10_7e61_40b7_935e_bedd4b1e481e.slice/crio-9e2c819b5a69276fd2b631b4d8f4a26c4ec6b65a392f71d33d1e502fbec2c1c7 WatchSource:0}: Error finding container 9e2c819b5a69276fd2b631b4d8f4a26c4ec6b65a392f71d33d1e502fbec2c1c7: Status 404 returned error can't find the container with id 9e2c819b5a69276fd2b631b4d8f4a26c4ec6b65a392f71d33d1e502fbec2c1c7 Sep 08 10:20:26.999555 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:26.999510 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" event={"ID":"f0e73e10-7e61-40b7-935e-bedd4b1e481e","Type":"ContainerStarted","Data":"9e2c819b5a69276fd2b631b4d8f4a26c4ec6b65a392f71d33d1e502fbec2c1c7"} Sep 08 10:20:29.009366 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.009338 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" event={"ID":"f0e73e10-7e61-40b7-935e-bedd4b1e481e","Type":"ContainerStarted","Data":"ed3abe090be098cb55c9d36b662783e994b37f5437030ab0cf816aa3f6d49773"} Sep 08 10:20:29.009662 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.009393 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" event={"ID":"f0e73e10-7e61-40b7-935e-bedd4b1e481e","Type":"ContainerStarted","Data":"e1bc6fcaa6cf9972aad114a20730c8f7589c375cd394c2a250c301ccd87b5ffb"} Sep 08 10:20:29.009662 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.009462 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" event={"ID":"f0e73e10-7e61-40b7-935e-bedd4b1e481e","Type":"ContainerStarted","Data":"3378629ef3e1a213c658872f630f4b071824d7a77e1531847d41c011e5075701"} Sep 08 10:20:29.039344 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.039294 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-54bf4db66-fst9v" podStartSLOduration=1.376010782 podStartE2EDuration="3.039274487s" totalImagesPullingTime="1.663263705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:20:29.038753876 +0000 UTC m=+205.408019177" watchObservedRunningTime="2026-09-08 10:20:29.039274487 +0000 UTC m=+205.408539785" Sep 08 10:20:29.204932 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.204902 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85c5dd787f-vjx8x"] Sep 08 10:20:29.208512 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.208485 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.245719 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.245666 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85c5dd787f-vjx8x"] Sep 08 10:20:29.249976 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.249954 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-oauth-config\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.250083 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.249987 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-trusted-ca-bundle\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.250083 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.250005 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-oauth-serving-cert\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.250083 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.250035 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx55z\" (UniqueName: \"kubernetes.io/projected/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-kube-api-access-wx55z\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.250188 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.250119 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-service-ca\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.250188 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.250174 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-serving-cert\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.250251 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.250192 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-config\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.351383 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.351309 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-service-ca\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.351383 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.351358 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-serving-cert\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.351383 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.351376 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-config\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.351678 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.351403 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-oauth-config\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.351678 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.351572 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-trusted-ca-bundle\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.351678 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.351615 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-oauth-serving-cert\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.351678 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.351652 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx55z\" (UniqueName: \"kubernetes.io/projected/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-kube-api-access-wx55z\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.352270 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.352228 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-service-ca\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.352270 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.352231 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-config\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.352413 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.352373 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-oauth-serving-cert\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.352650 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.352627 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-trusted-ca-bundle\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.353933 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.353909 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-serving-cert\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.354076 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.354056 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-oauth-config\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.362622 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.360870 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx55z\" (UniqueName: \"kubernetes.io/projected/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-kube-api-access-wx55z\") pod \"console-85c5dd787f-vjx8x\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.518840 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.518808 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:29.813074 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:29.813052 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85c5dd787f-vjx8x"] Sep 08 10:20:29.815229 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:20:29.815202 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff37a4f4_c333_42f6_85f9_2f3b0cb32034.slice/crio-022d8e9b54bb14646383144bc35f9e5b5964035dfbcee0056951f9b8144739ec WatchSource:0}: Error finding container 022d8e9b54bb14646383144bc35f9e5b5964035dfbcee0056951f9b8144739ec: Status 404 returned error can't find the container with id 022d8e9b54bb14646383144bc35f9e5b5964035dfbcee0056951f9b8144739ec Sep 08 10:20:30.017149 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:30.017117 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85c5dd787f-vjx8x" event={"ID":"ff37a4f4-c333-42f6-85f9-2f3b0cb32034","Type":"ContainerStarted","Data":"8ed4317b420c5180831c971e7894b043bad7c98db113d1330c0eb81f602a4344"} Sep 08 10:20:30.017554 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:30.017165 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85c5dd787f-vjx8x" event={"ID":"ff37a4f4-c333-42f6-85f9-2f3b0cb32034","Type":"ContainerStarted","Data":"022d8e9b54bb14646383144bc35f9e5b5964035dfbcee0056951f9b8144739ec"} Sep 08 10:20:30.038719 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:30.038671 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85c5dd787f-vjx8x" podStartSLOduration=1.038655135 podStartE2EDuration="1.038655135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:20:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:20:30.037134896 +0000 UTC m=+206.406400192" watchObservedRunningTime="2026-09-08 10:20:30.038655135 +0000 UTC m=+206.407920431" Sep 08 10:20:38.816625 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:38.816594 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85c5dd787f-vjx8x"] Sep 08 10:20:38.855817 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:38.855788 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c6867f55f-sdggf"] Sep 08 10:20:38.861913 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:38.861899 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:38.870841 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:38.870819 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c6867f55f-sdggf"] Sep 08 10:20:38.923847 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:38.923823 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv7tf\" (UniqueName: \"kubernetes.io/projected/be32b4f1-dd72-4485-8dac-d425bffac868-kube-api-access-rv7tf\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:38.923953 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:38.923853 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-console-config\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:38.923953 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:38.923877 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-service-ca\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:38.923953 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:38.923921 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-trusted-ca-bundle\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:38.924057 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:38.923987 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-oauth-serving-cert\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:38.924057 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:38.924012 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be32b4f1-dd72-4485-8dac-d425bffac868-console-oauth-config\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:38.924057 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:38.924039 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be32b4f1-dd72-4485-8dac-d425bffac868-console-serving-cert\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:39.025310 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.025282 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-service-ca\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:39.025310 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.025311 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-trusted-ca-bundle\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:39.025443 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.025337 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-oauth-serving-cert\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:39.025482 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.025445 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be32b4f1-dd72-4485-8dac-d425bffac868-console-oauth-config\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:39.025516 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.025494 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be32b4f1-dd72-4485-8dac-d425bffac868-console-serving-cert\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:39.025579 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.025564 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv7tf\" (UniqueName: \"kubernetes.io/projected/be32b4f1-dd72-4485-8dac-d425bffac868-kube-api-access-rv7tf\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:39.025620 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.025592 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-console-config\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:39.026073 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.026046 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-service-ca\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:39.026182 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.026166 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-oauth-serving-cert\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:39.026235 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.026202 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-trusted-ca-bundle\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:39.026235 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.026221 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-console-config\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:39.028458 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.028435 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be32b4f1-dd72-4485-8dac-d425bffac868-console-serving-cert\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:39.028562 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.028454 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be32b4f1-dd72-4485-8dac-d425bffac868-console-oauth-config\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:39.034069 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.034050 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv7tf\" (UniqueName: \"kubernetes.io/projected/be32b4f1-dd72-4485-8dac-d425bffac868-kube-api-access-rv7tf\") pod \"console-c6867f55f-sdggf\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:39.170900 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.170825 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:39.446903 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.446873 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c6867f55f-sdggf"] Sep 08 10:20:39.449682 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:20:39.449656 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe32b4f1_dd72_4485_8dac_d425bffac868.slice/crio-d446151d84b1c0a33d32e4c207d0d2a55e1affd953ed864a57c162eb944b3d74 WatchSource:0}: Error finding container d446151d84b1c0a33d32e4c207d0d2a55e1affd953ed864a57c162eb944b3d74: Status 404 returned error can't find the container with id d446151d84b1c0a33d32e4c207d0d2a55e1affd953ed864a57c162eb944b3d74 Sep 08 10:20:39.519504 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:39.519479 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:20:40.057815 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:40.057781 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6867f55f-sdggf" event={"ID":"be32b4f1-dd72-4485-8dac-d425bffac868","Type":"ContainerStarted","Data":"6c9e2c75a6b1a22a32f2a12c113832fa892801ece6868a161ce3c9f3e1eccc09"} Sep 08 10:20:40.058166 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:40.057836 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6867f55f-sdggf" event={"ID":"be32b4f1-dd72-4485-8dac-d425bffac868","Type":"ContainerStarted","Data":"d446151d84b1c0a33d32e4c207d0d2a55e1affd953ed864a57c162eb944b3d74"} Sep 08 10:20:40.085591 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:40.085519 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c6867f55f-sdggf" podStartSLOduration=2.085503171 podStartE2EDuration="2.085503171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:20:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:20:40.08421949 +0000 UTC m=+216.453484787" watchObservedRunningTime="2026-09-08 10:20:40.085503171 +0000 UTC m=+216.454768468" Sep 08 10:20:49.171489 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:49.171458 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:49.172037 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:49.171508 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:49.175897 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:49.175877 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:50.105933 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:50.105904 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:20:50.163741 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:20:50.163705 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-695f789945-fz72g"] Sep 08 10:21:03.835580 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:03.835521 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85c5dd787f-vjx8x" podUID="ff37a4f4-c333-42f6-85f9-2f3b0cb32034" containerName="console" containerID="cri-o://8ed4317b420c5180831c971e7894b043bad7c98db113d1330c0eb81f602a4344" gracePeriod=15 Sep 08 10:21:04.079637 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.079611 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:21:04.160075 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.160003 2561 generic.go:356] "Generic (PLEG): container finished" podID="ff37a4f4-c333-42f6-85f9-2f3b0cb32034" containerID="8ed4317b420c5180831c971e7894b043bad7c98db113d1330c0eb81f602a4344" exitCode=0 Sep 08 10:21:04.160075 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.160048 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85c5dd787f-vjx8x" Sep 08 10:21:04.160075 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.160043 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85c5dd787f-vjx8x" event={"ID":"ff37a4f4-c333-42f6-85f9-2f3b0cb32034","Type":"ContainerDied","Data":"8ed4317b420c5180831c971e7894b043bad7c98db113d1330c0eb81f602a4344"} Sep 08 10:21:04.160310 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.160098 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85c5dd787f-vjx8x" event={"ID":"ff37a4f4-c333-42f6-85f9-2f3b0cb32034","Type":"ContainerDied","Data":"022d8e9b54bb14646383144bc35f9e5b5964035dfbcee0056951f9b8144739ec"} Sep 08 10:21:04.160310 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.160122 2561 scope.go:117] "RemoveContainer" containerID="8ed4317b420c5180831c971e7894b043bad7c98db113d1330c0eb81f602a4344" Sep 08 10:21:04.168704 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.168687 2561 scope.go:117] "RemoveContainer" containerID="8ed4317b420c5180831c971e7894b043bad7c98db113d1330c0eb81f602a4344" Sep 08 10:21:04.168943 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:21:04.168927 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ed4317b420c5180831c971e7894b043bad7c98db113d1330c0eb81f602a4344\": container with ID starting with 8ed4317b420c5180831c971e7894b043bad7c98db113d1330c0eb81f602a4344 not found: ID does not exist" containerID="8ed4317b420c5180831c971e7894b043bad7c98db113d1330c0eb81f602a4344" Sep 08 10:21:04.168998 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.168953 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ed4317b420c5180831c971e7894b043bad7c98db113d1330c0eb81f602a4344"} err="failed to get container status \"8ed4317b420c5180831c971e7894b043bad7c98db113d1330c0eb81f602a4344\": rpc error: code = NotFound desc = could not find container \"8ed4317b420c5180831c971e7894b043bad7c98db113d1330c0eb81f602a4344\": container with ID starting with 8ed4317b420c5180831c971e7894b043bad7c98db113d1330c0eb81f602a4344 not found: ID does not exist" Sep 08 10:21:04.234653 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.234632 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-trusted-ca-bundle\") pod \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " Sep 08 10:21:04.234757 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.234661 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-service-ca\") pod \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " Sep 08 10:21:04.234757 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.234691 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wx55z\" (UniqueName: \"kubernetes.io/projected/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-kube-api-access-wx55z\") pod \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " Sep 08 10:21:04.234757 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.234732 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-oauth-serving-cert\") pod \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " Sep 08 10:21:04.234909 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.234755 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-config\") pod \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " Sep 08 10:21:04.234909 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.234843 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-serving-cert\") pod \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " Sep 08 10:21:04.234997 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.234908 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-oauth-config\") pod \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\" (UID: \"ff37a4f4-c333-42f6-85f9-2f3b0cb32034\") " Sep 08 10:21:04.235079 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.235040 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ff37a4f4-c333-42f6-85f9-2f3b0cb32034" (UID: "ff37a4f4-c333-42f6-85f9-2f3b0cb32034"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:21:04.235211 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.235185 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-service-ca" (OuterVolumeSpecName: "service-ca") pod "ff37a4f4-c333-42f6-85f9-2f3b0cb32034" (UID: "ff37a4f4-c333-42f6-85f9-2f3b0cb32034"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:21:04.235347 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.235325 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ff37a4f4-c333-42f6-85f9-2f3b0cb32034" (UID: "ff37a4f4-c333-42f6-85f9-2f3b0cb32034"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:21:04.235347 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.235338 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-config" (OuterVolumeSpecName: "console-config") pod "ff37a4f4-c333-42f6-85f9-2f3b0cb32034" (UID: "ff37a4f4-c333-42f6-85f9-2f3b0cb32034"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:21:04.236791 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.236763 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-kube-api-access-wx55z" (OuterVolumeSpecName: "kube-api-access-wx55z") pod "ff37a4f4-c333-42f6-85f9-2f3b0cb32034" (UID: "ff37a4f4-c333-42f6-85f9-2f3b0cb32034"). InnerVolumeSpecName "kube-api-access-wx55z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:21:04.236878 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.236823 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ff37a4f4-c333-42f6-85f9-2f3b0cb32034" (UID: "ff37a4f4-c333-42f6-85f9-2f3b0cb32034"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:21:04.236951 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.236925 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ff37a4f4-c333-42f6-85f9-2f3b0cb32034" (UID: "ff37a4f4-c333-42f6-85f9-2f3b0cb32034"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:21:04.335571 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.335543 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-serving-cert\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:21:04.335571 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.335568 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-oauth-config\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:21:04.335711 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.335582 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-trusted-ca-bundle\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:21:04.335711 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.335595 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-service-ca\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:21:04.335711 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.335607 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx55z\" (UniqueName: \"kubernetes.io/projected/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-kube-api-access-wx55z\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:21:04.335711 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.335620 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-oauth-serving-cert\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:21:04.335711 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.335631 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff37a4f4-c333-42f6-85f9-2f3b0cb32034-console-config\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:21:04.480306 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.480281 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85c5dd787f-vjx8x"] Sep 08 10:21:04.485806 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:04.485766 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85c5dd787f-vjx8x"] Sep 08 10:21:06.144900 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:06.144865 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff37a4f4-c333-42f6-85f9-2f3b0cb32034" path="/var/lib/kubelet/pods/ff37a4f4-c333-42f6-85f9-2f3b0cb32034/volumes" Sep 08 10:21:15.185635 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.185578 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-695f789945-fz72g" podUID="43038ed5-4a97-46d3-aaa1-c43a61327e22" containerName="console" containerID="cri-o://b26c3999efb29a905547eb95c1244ebc9f1d840702cc99505b2a2036f585eca7" gracePeriod=15 Sep 08 10:21:15.425427 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.425400 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695f789945-fz72g" Sep 08 10:21:15.430958 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.430935 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-service-ca\") pod \"43038ed5-4a97-46d3-aaa1-c43a61327e22\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " Sep 08 10:21:15.431064 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.430980 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-serving-cert\") pod \"43038ed5-4a97-46d3-aaa1-c43a61327e22\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " Sep 08 10:21:15.431064 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.431009 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-trusted-ca-bundle\") pod \"43038ed5-4a97-46d3-aaa1-c43a61327e22\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " Sep 08 10:21:15.431064 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.431057 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-oauth-config\") pod \"43038ed5-4a97-46d3-aaa1-c43a61327e22\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " Sep 08 10:21:15.431223 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.431081 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lz6mf\" (UniqueName: \"kubernetes.io/projected/43038ed5-4a97-46d3-aaa1-c43a61327e22-kube-api-access-lz6mf\") pod \"43038ed5-4a97-46d3-aaa1-c43a61327e22\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " Sep 08 10:21:15.431223 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.431122 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-oauth-serving-cert\") pod \"43038ed5-4a97-46d3-aaa1-c43a61327e22\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " Sep 08 10:21:15.431223 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.431159 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-config\") pod \"43038ed5-4a97-46d3-aaa1-c43a61327e22\" (UID: \"43038ed5-4a97-46d3-aaa1-c43a61327e22\") " Sep 08 10:21:15.431382 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.431356 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-service-ca" (OuterVolumeSpecName: "service-ca") pod "43038ed5-4a97-46d3-aaa1-c43a61327e22" (UID: "43038ed5-4a97-46d3-aaa1-c43a61327e22"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:21:15.431575 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.431521 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "43038ed5-4a97-46d3-aaa1-c43a61327e22" (UID: "43038ed5-4a97-46d3-aaa1-c43a61327e22"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:21:15.431676 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.431601 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "43038ed5-4a97-46d3-aaa1-c43a61327e22" (UID: "43038ed5-4a97-46d3-aaa1-c43a61327e22"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:21:15.431724 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.431671 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-config" (OuterVolumeSpecName: "console-config") pod "43038ed5-4a97-46d3-aaa1-c43a61327e22" (UID: "43038ed5-4a97-46d3-aaa1-c43a61327e22"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:21:15.433128 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.433104 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "43038ed5-4a97-46d3-aaa1-c43a61327e22" (UID: "43038ed5-4a97-46d3-aaa1-c43a61327e22"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:21:15.433226 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.433156 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "43038ed5-4a97-46d3-aaa1-c43a61327e22" (UID: "43038ed5-4a97-46d3-aaa1-c43a61327e22"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:21:15.433226 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.433169 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43038ed5-4a97-46d3-aaa1-c43a61327e22-kube-api-access-lz6mf" (OuterVolumeSpecName: "kube-api-access-lz6mf") pod "43038ed5-4a97-46d3-aaa1-c43a61327e22" (UID: "43038ed5-4a97-46d3-aaa1-c43a61327e22"). InnerVolumeSpecName "kube-api-access-lz6mf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:21:15.531913 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.531874 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-service-ca\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:21:15.531913 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.531904 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-serving-cert\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:21:15.531913 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.531916 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-trusted-ca-bundle\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:21:15.531913 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.531925 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-oauth-config\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:21:15.532165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.531935 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lz6mf\" (UniqueName: \"kubernetes.io/projected/43038ed5-4a97-46d3-aaa1-c43a61327e22-kube-api-access-lz6mf\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:21:15.532165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.531945 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-oauth-serving-cert\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:21:15.532165 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:15.531953 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43038ed5-4a97-46d3-aaa1-c43a61327e22-console-config\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:21:16.213698 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:16.213662 2561 generic.go:356] "Generic (PLEG): container finished" podID="43038ed5-4a97-46d3-aaa1-c43a61327e22" containerID="b26c3999efb29a905547eb95c1244ebc9f1d840702cc99505b2a2036f585eca7" exitCode=0 Sep 08 10:21:16.214099 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:16.213703 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695f789945-fz72g" event={"ID":"43038ed5-4a97-46d3-aaa1-c43a61327e22","Type":"ContainerDied","Data":"b26c3999efb29a905547eb95c1244ebc9f1d840702cc99505b2a2036f585eca7"} Sep 08 10:21:16.214099 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:16.213707 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695f789945-fz72g" Sep 08 10:21:16.214099 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:16.213749 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695f789945-fz72g" event={"ID":"43038ed5-4a97-46d3-aaa1-c43a61327e22","Type":"ContainerDied","Data":"12accf1e3622091a808b1de684e4a125986d1747990d05fd28801e3cacfd33ef"} Sep 08 10:21:16.214099 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:16.213770 2561 scope.go:117] "RemoveContainer" containerID="b26c3999efb29a905547eb95c1244ebc9f1d840702cc99505b2a2036f585eca7" Sep 08 10:21:16.222022 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:16.222004 2561 scope.go:117] "RemoveContainer" containerID="b26c3999efb29a905547eb95c1244ebc9f1d840702cc99505b2a2036f585eca7" Sep 08 10:21:16.222315 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:21:16.222294 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b26c3999efb29a905547eb95c1244ebc9f1d840702cc99505b2a2036f585eca7\": container with ID starting with b26c3999efb29a905547eb95c1244ebc9f1d840702cc99505b2a2036f585eca7 not found: ID does not exist" containerID="b26c3999efb29a905547eb95c1244ebc9f1d840702cc99505b2a2036f585eca7" Sep 08 10:21:16.222357 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:16.222325 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b26c3999efb29a905547eb95c1244ebc9f1d840702cc99505b2a2036f585eca7"} err="failed to get container status \"b26c3999efb29a905547eb95c1244ebc9f1d840702cc99505b2a2036f585eca7\": rpc error: code = NotFound desc = could not find container \"b26c3999efb29a905547eb95c1244ebc9f1d840702cc99505b2a2036f585eca7\": container with ID starting with b26c3999efb29a905547eb95c1244ebc9f1d840702cc99505b2a2036f585eca7 not found: ID does not exist" Sep 08 10:21:16.236894 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:16.236865 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-695f789945-fz72g"] Sep 08 10:21:16.245982 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:16.245956 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-695f789945-fz72g"] Sep 08 10:21:18.144874 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:21:18.144842 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43038ed5-4a97-46d3-aaa1-c43a61327e22" path="/var/lib/kubelet/pods/43038ed5-4a97-46d3-aaa1-c43a61327e22/volumes" Sep 08 10:22:04.063159 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:22:04.063130 2561 kubelet.go:1648] "Image garbage collection succeeded" Sep 08 10:23:10.924741 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:10.924661 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq"] Sep 08 10:23:10.925146 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:10.924981 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43038ed5-4a97-46d3-aaa1-c43a61327e22" containerName="console" Sep 08 10:23:10.925146 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:10.924992 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="43038ed5-4a97-46d3-aaa1-c43a61327e22" containerName="console" Sep 08 10:23:10.925146 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:10.925002 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff37a4f4-c333-42f6-85f9-2f3b0cb32034" containerName="console" Sep 08 10:23:10.925146 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:10.925009 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff37a4f4-c333-42f6-85f9-2f3b0cb32034" containerName="console" Sep 08 10:23:10.925146 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:10.925063 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="43038ed5-4a97-46d3-aaa1-c43a61327e22" containerName="console" Sep 08 10:23:10.925146 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:10.925071 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff37a4f4-c333-42f6-85f9-2f3b0cb32034" containerName="console" Sep 08 10:23:10.927962 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:10.927944 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq" Sep 08 10:23:10.930102 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:10.930077 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 08 10:23:10.930778 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:10.930763 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-d29w4\"" Sep 08 10:23:10.930842 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:10.930775 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 08 10:23:10.947128 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:10.947096 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq"] Sep 08 10:23:11.041504 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:11.041472 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v665z\" (UniqueName: \"kubernetes.io/projected/81540954-a1ae-44d3-88b5-43f2d0c9d186-kube-api-access-v665z\") pod \"feast-operator-controller-manager-5bc89c8fcd-d7knq\" (UID: \"81540954-a1ae-44d3-88b5-43f2d0c9d186\") " pod="feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq" Sep 08 10:23:11.142016 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:11.141991 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v665z\" (UniqueName: \"kubernetes.io/projected/81540954-a1ae-44d3-88b5-43f2d0c9d186-kube-api-access-v665z\") pod \"feast-operator-controller-manager-5bc89c8fcd-d7knq\" (UID: \"81540954-a1ae-44d3-88b5-43f2d0c9d186\") " pod="feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq" Sep 08 10:23:11.152197 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:11.152167 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v665z\" (UniqueName: \"kubernetes.io/projected/81540954-a1ae-44d3-88b5-43f2d0c9d186-kube-api-access-v665z\") pod \"feast-operator-controller-manager-5bc89c8fcd-d7knq\" (UID: \"81540954-a1ae-44d3-88b5-43f2d0c9d186\") " pod="feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq" Sep 08 10:23:11.238671 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:11.238648 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq" Sep 08 10:23:11.516343 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:11.516317 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq"] Sep 08 10:23:11.518823 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:23:11.518798 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81540954_a1ae_44d3_88b5_43f2d0c9d186.slice/crio-9ccd118cdd482c6aa63fd70f76a12629b6b528c09ff6e0e74c919d39861ef06c WatchSource:0}: Error finding container 9ccd118cdd482c6aa63fd70f76a12629b6b528c09ff6e0e74c919d39861ef06c: Status 404 returned error can't find the container with id 9ccd118cdd482c6aa63fd70f76a12629b6b528c09ff6e0e74c919d39861ef06c Sep 08 10:23:11.520712 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:11.520693 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 10:23:11.657588 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:11.657546 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq" event={"ID":"81540954-a1ae-44d3-88b5-43f2d0c9d186","Type":"ContainerStarted","Data":"9ccd118cdd482c6aa63fd70f76a12629b6b528c09ff6e0e74c919d39861ef06c"} Sep 08 10:23:14.671869 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:14.671829 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq" event={"ID":"81540954-a1ae-44d3-88b5-43f2d0c9d186","Type":"ContainerStarted","Data":"411173ee6da70811fc18439f9ed09d95a2bc003e6cf553a0b12c06e0b54639f2"} Sep 08 10:23:14.672233 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:14.671895 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq" Sep 08 10:23:14.702430 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:14.702388 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq" podStartSLOduration=2.138022021 podStartE2EDuration="4.702373788s" totalImagesPullingTime="2.564351767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:23:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:23:14.702046382 +0000 UTC m=+371.071311678" watchObservedRunningTime="2026-09-08 10:23:14.702373788 +0000 UTC m=+371.071639085" Sep 08 10:23:25.678464 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:25.678429 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq" Sep 08 10:23:42.996289 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:42.996249 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d858685c7-vr45p"] Sep 08 10:23:42.999894 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:42.999866 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.014628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.014600 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d858685c7-vr45p"] Sep 08 10:23:43.078519 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.078485 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/755502c6-7937-4732-91a3-1ccf11d99aae-oauth-serving-cert\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.078673 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.078571 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/755502c6-7937-4732-91a3-1ccf11d99aae-console-serving-cert\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.078673 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.078611 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcrq6\" (UniqueName: \"kubernetes.io/projected/755502c6-7937-4732-91a3-1ccf11d99aae-kube-api-access-bcrq6\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.078673 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.078644 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/755502c6-7937-4732-91a3-1ccf11d99aae-trusted-ca-bundle\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.078847 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.078681 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/755502c6-7937-4732-91a3-1ccf11d99aae-console-config\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.078847 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.078750 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/755502c6-7937-4732-91a3-1ccf11d99aae-service-ca\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.078847 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.078782 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/755502c6-7937-4732-91a3-1ccf11d99aae-console-oauth-config\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.179625 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.179601 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/755502c6-7937-4732-91a3-1ccf11d99aae-trusted-ca-bundle\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.179735 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.179633 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/755502c6-7937-4732-91a3-1ccf11d99aae-console-config\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.179735 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.179659 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/755502c6-7937-4732-91a3-1ccf11d99aae-service-ca\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.179735 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.179678 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/755502c6-7937-4732-91a3-1ccf11d99aae-console-oauth-config\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.179735 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.179708 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/755502c6-7937-4732-91a3-1ccf11d99aae-oauth-serving-cert\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.179928 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.179742 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/755502c6-7937-4732-91a3-1ccf11d99aae-console-serving-cert\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.179928 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.179757 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcrq6\" (UniqueName: \"kubernetes.io/projected/755502c6-7937-4732-91a3-1ccf11d99aae-kube-api-access-bcrq6\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.180470 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.180446 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/755502c6-7937-4732-91a3-1ccf11d99aae-service-ca\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.180593 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.180506 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/755502c6-7937-4732-91a3-1ccf11d99aae-oauth-serving-cert\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.180759 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.180737 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/755502c6-7937-4732-91a3-1ccf11d99aae-console-config\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.180821 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.180802 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/755502c6-7937-4732-91a3-1ccf11d99aae-trusted-ca-bundle\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.182296 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.182274 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/755502c6-7937-4732-91a3-1ccf11d99aae-console-oauth-config\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.182431 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.182409 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/755502c6-7937-4732-91a3-1ccf11d99aae-console-serving-cert\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.189603 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.189577 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcrq6\" (UniqueName: \"kubernetes.io/projected/755502c6-7937-4732-91a3-1ccf11d99aae-kube-api-access-bcrq6\") pod \"console-5d858685c7-vr45p\" (UID: \"755502c6-7937-4732-91a3-1ccf11d99aae\") " pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.310563 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.310495 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:43.583423 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.583396 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d858685c7-vr45p"] Sep 08 10:23:43.585911 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:23:43.585865 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod755502c6_7937_4732_91a3_1ccf11d99aae.slice/crio-a8a5f7a940a5932750d8db73bbec5f28c3501d45e30dfd966fa68f35dcc8dbb9 WatchSource:0}: Error finding container a8a5f7a940a5932750d8db73bbec5f28c3501d45e30dfd966fa68f35dcc8dbb9: Status 404 returned error can't find the container with id a8a5f7a940a5932750d8db73bbec5f28c3501d45e30dfd966fa68f35dcc8dbb9 Sep 08 10:23:43.790603 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.790565 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d858685c7-vr45p" event={"ID":"755502c6-7937-4732-91a3-1ccf11d99aae","Type":"ContainerStarted","Data":"4156eef2ff88917fc3c4271d111f1b3666118ec1d070d60c7e54e10d805d128e"} Sep 08 10:23:43.790751 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.790622 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d858685c7-vr45p" event={"ID":"755502c6-7937-4732-91a3-1ccf11d99aae","Type":"ContainerStarted","Data":"a8a5f7a940a5932750d8db73bbec5f28c3501d45e30dfd966fa68f35dcc8dbb9"} Sep 08 10:23:43.813342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:43.813291 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d858685c7-vr45p" podStartSLOduration=1.813275083 podStartE2EDuration="1.813275083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:23:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:23:43.810569166 +0000 UTC m=+400.179834460" watchObservedRunningTime="2026-09-08 10:23:43.813275083 +0000 UTC m=+400.182540379" Sep 08 10:23:53.311307 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.311257 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:53.311721 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.311330 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:53.316185 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.316160 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:53.836343 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.836314 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d858685c7-vr45p" Sep 08 10:23:53.894201 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.894171 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c6867f55f-sdggf"] Sep 08 10:23:53.905753 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.905719 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4"] Sep 08 10:23:53.913047 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.913019 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:53.917053 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.917031 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 08 10:23:53.917210 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.917056 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 08 10:23:53.917304 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.917285 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 08 10:23:53.917368 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.917332 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-r9sgg\"" Sep 08 10:23:53.917368 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.917081 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 08 10:23:53.917487 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.917099 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 08 10:23:53.917487 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.917121 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 08 10:23:53.917487 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.917287 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 08 10:23:53.926551 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.926505 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4"] Sep 08 10:23:53.964464 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.964438 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b4df702c-8acd-49f0-811a-dba71704df24-feast-data\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:53.964592 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.964473 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-online-tls\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:53.964592 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.964494 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qll4p\" (UniqueName: \"kubernetes.io/projected/b4df702c-8acd-49f0-811a-dba71704df24-kube-api-access-qll4p\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:53.964592 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.964512 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-ui-tls\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:53.964714 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.964651 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-offline-tls\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:53.964752 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:53.964732 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-registry-tls\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:54.066014 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:54.065985 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-offline-tls\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:54.066123 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:54.066026 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-registry-tls\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:54.066123 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:54.066053 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b4df702c-8acd-49f0-811a-dba71704df24-feast-data\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:54.066123 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:54.066082 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-online-tls\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:54.066123 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:54.066115 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qll4p\" (UniqueName: \"kubernetes.io/projected/b4df702c-8acd-49f0-811a-dba71704df24-kube-api-access-qll4p\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:54.066299 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:23:54.066125 2561 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 08 10:23:54.066299 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:54.066142 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-ui-tls\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:54.066299 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:23:54.066193 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-offline-tls podName:b4df702c-8acd-49f0-811a-dba71704df24 nodeName:}" failed. No retries permitted until 2026-09-08 10:23:54.56617309 +0000 UTC m=+410.935438366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-offline-tls") pod "feast-simple-feast-setup-8456cc85bf-t5zc4" (UID: "b4df702c-8acd-49f0-811a-dba71704df24") : secret "feast-simple-feast-setup-offline-tls" not found Sep 08 10:23:54.066449 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:54.066435 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b4df702c-8acd-49f0-811a-dba71704df24-feast-data\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:54.068523 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:54.068499 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-online-tls\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:54.068630 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:54.068516 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-registry-tls\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:54.068630 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:54.068621 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-ui-tls\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:54.076615 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:54.076592 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qll4p\" (UniqueName: \"kubernetes.io/projected/b4df702c-8acd-49f0-811a-dba71704df24-kube-api-access-qll4p\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:54.570824 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:54.570789 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-offline-tls\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:54.573147 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:54.573126 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-offline-tls\") pod \"feast-simple-feast-setup-8456cc85bf-t5zc4\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:54.830183 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:54.830101 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:23:55.114811 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:55.114787 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4"] Sep 08 10:23:55.116614 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:23:55.116588 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4df702c_8acd_49f0_811a_dba71704df24.slice/crio-02f4a46265f6662e9ab0318513768b30498c6fdafaaf2dd7e66c4cb4c76b95c4 WatchSource:0}: Error finding container 02f4a46265f6662e9ab0318513768b30498c6fdafaaf2dd7e66c4cb4c76b95c4: Status 404 returned error can't find the container with id 02f4a46265f6662e9ab0318513768b30498c6fdafaaf2dd7e66c4cb4c76b95c4 Sep 08 10:23:55.841205 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:23:55.841164 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" event={"ID":"b4df702c-8acd-49f0-811a-dba71704df24","Type":"ContainerStarted","Data":"02f4a46265f6662e9ab0318513768b30498c6fdafaaf2dd7e66c4cb4c76b95c4"} Sep 08 10:24:18.919577 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:18.919518 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c6867f55f-sdggf" podUID="be32b4f1-dd72-4485-8dac-d425bffac868" containerName="console" containerID="cri-o://6c9e2c75a6b1a22a32f2a12c113832fa892801ece6868a161ce3c9f3e1eccc09" gracePeriod=15 Sep 08 10:24:20.102872 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:20.102822 2561 patch_prober.go:28] interesting pod/console-c6867f55f-sdggf container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.132.0.24:8443/health\": dial tcp 10.132.0.24:8443: connect: connection refused" start-of-body= Sep 08 10:24:20.103301 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:20.102902 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-c6867f55f-sdggf" podUID="be32b4f1-dd72-4485-8dac-d425bffac868" containerName="console" probeResult="failure" output="Get \"https://10.132.0.24:8443/health\": dial tcp 10.132.0.24:8443: connect: connection refused" Sep 08 10:24:21.728235 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.728208 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:24:21.819190 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.819163 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rv7tf\" (UniqueName: \"kubernetes.io/projected/be32b4f1-dd72-4485-8dac-d425bffac868-kube-api-access-rv7tf\") pod \"be32b4f1-dd72-4485-8dac-d425bffac868\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " Sep 08 10:24:21.819311 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.819221 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be32b4f1-dd72-4485-8dac-d425bffac868-console-oauth-config\") pod \"be32b4f1-dd72-4485-8dac-d425bffac868\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " Sep 08 10:24:21.819311 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.819266 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-oauth-serving-cert\") pod \"be32b4f1-dd72-4485-8dac-d425bffac868\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " Sep 08 10:24:21.819405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.819310 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-console-config\") pod \"be32b4f1-dd72-4485-8dac-d425bffac868\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " Sep 08 10:24:21.819405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.819329 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be32b4f1-dd72-4485-8dac-d425bffac868-console-serving-cert\") pod \"be32b4f1-dd72-4485-8dac-d425bffac868\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " Sep 08 10:24:21.819405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.819353 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-trusted-ca-bundle\") pod \"be32b4f1-dd72-4485-8dac-d425bffac868\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " Sep 08 10:24:21.819405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.819383 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-service-ca\") pod \"be32b4f1-dd72-4485-8dac-d425bffac868\" (UID: \"be32b4f1-dd72-4485-8dac-d425bffac868\") " Sep 08 10:24:21.819882 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.819740 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "be32b4f1-dd72-4485-8dac-d425bffac868" (UID: "be32b4f1-dd72-4485-8dac-d425bffac868"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:24:21.819882 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.819835 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-console-config" (OuterVolumeSpecName: "console-config") pod "be32b4f1-dd72-4485-8dac-d425bffac868" (UID: "be32b4f1-dd72-4485-8dac-d425bffac868"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:24:21.819882 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.819863 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "be32b4f1-dd72-4485-8dac-d425bffac868" (UID: "be32b4f1-dd72-4485-8dac-d425bffac868"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:24:21.820103 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.819913 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-service-ca" (OuterVolumeSpecName: "service-ca") pod "be32b4f1-dd72-4485-8dac-d425bffac868" (UID: "be32b4f1-dd72-4485-8dac-d425bffac868"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:24:21.821366 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.821332 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be32b4f1-dd72-4485-8dac-d425bffac868-kube-api-access-rv7tf" (OuterVolumeSpecName: "kube-api-access-rv7tf") pod "be32b4f1-dd72-4485-8dac-d425bffac868" (UID: "be32b4f1-dd72-4485-8dac-d425bffac868"). InnerVolumeSpecName "kube-api-access-rv7tf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:24:21.821451 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.821363 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be32b4f1-dd72-4485-8dac-d425bffac868-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "be32b4f1-dd72-4485-8dac-d425bffac868" (UID: "be32b4f1-dd72-4485-8dac-d425bffac868"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:24:21.821451 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.821407 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be32b4f1-dd72-4485-8dac-d425bffac868-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "be32b4f1-dd72-4485-8dac-d425bffac868" (UID: "be32b4f1-dd72-4485-8dac-d425bffac868"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:24:21.920084 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.920054 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-service-ca\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:24:21.920170 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.920085 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rv7tf\" (UniqueName: \"kubernetes.io/projected/be32b4f1-dd72-4485-8dac-d425bffac868-kube-api-access-rv7tf\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:24:21.920170 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.920099 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be32b4f1-dd72-4485-8dac-d425bffac868-console-oauth-config\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:24:21.920170 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.920115 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-oauth-serving-cert\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:24:21.920170 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.920128 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-console-config\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:24:21.920170 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.920141 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be32b4f1-dd72-4485-8dac-d425bffac868-console-serving-cert\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:24:21.920170 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.920154 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be32b4f1-dd72-4485-8dac-d425bffac868-trusted-ca-bundle\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:24:21.967650 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.966922 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" event={"ID":"b4df702c-8acd-49f0-811a-dba71704df24","Type":"ContainerStarted","Data":"ff56ee8b94923bea52fbd4916ca309ae17e134cfeedc69bdf12386c77c9c02a2"} Sep 08 10:24:21.973210 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.969081 2561 generic.go:356] "Generic (PLEG): container finished" podID="be32b4f1-dd72-4485-8dac-d425bffac868" containerID="6c9e2c75a6b1a22a32f2a12c113832fa892801ece6868a161ce3c9f3e1eccc09" exitCode=0 Sep 08 10:24:21.973210 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.969129 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6867f55f-sdggf" event={"ID":"be32b4f1-dd72-4485-8dac-d425bffac868","Type":"ContainerDied","Data":"6c9e2c75a6b1a22a32f2a12c113832fa892801ece6868a161ce3c9f3e1eccc09"} Sep 08 10:24:21.973210 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.969168 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6867f55f-sdggf" event={"ID":"be32b4f1-dd72-4485-8dac-d425bffac868","Type":"ContainerDied","Data":"d446151d84b1c0a33d32e4c207d0d2a55e1affd953ed864a57c162eb944b3d74"} Sep 08 10:24:21.973210 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.969193 2561 scope.go:117] "RemoveContainer" containerID="6c9e2c75a6b1a22a32f2a12c113832fa892801ece6868a161ce3c9f3e1eccc09" Sep 08 10:24:21.973210 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.969334 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6867f55f-sdggf" Sep 08 10:24:21.990878 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.989611 2561 scope.go:117] "RemoveContainer" containerID="6c9e2c75a6b1a22a32f2a12c113832fa892801ece6868a161ce3c9f3e1eccc09" Sep 08 10:24:21.990878 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:24:21.989903 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c9e2c75a6b1a22a32f2a12c113832fa892801ece6868a161ce3c9f3e1eccc09\": container with ID starting with 6c9e2c75a6b1a22a32f2a12c113832fa892801ece6868a161ce3c9f3e1eccc09 not found: ID does not exist" containerID="6c9e2c75a6b1a22a32f2a12c113832fa892801ece6868a161ce3c9f3e1eccc09" Sep 08 10:24:21.990878 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:21.989938 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c9e2c75a6b1a22a32f2a12c113832fa892801ece6868a161ce3c9f3e1eccc09"} err="failed to get container status \"6c9e2c75a6b1a22a32f2a12c113832fa892801ece6868a161ce3c9f3e1eccc09\": rpc error: code = NotFound desc = could not find container \"6c9e2c75a6b1a22a32f2a12c113832fa892801ece6868a161ce3c9f3e1eccc09\": container with ID starting with 6c9e2c75a6b1a22a32f2a12c113832fa892801ece6868a161ce3c9f3e1eccc09 not found: ID does not exist" Sep 08 10:24:22.018182 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:22.018150 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c6867f55f-sdggf"] Sep 08 10:24:22.020986 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:22.020963 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c6867f55f-sdggf"] Sep 08 10:24:22.151200 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:22.151129 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be32b4f1-dd72-4485-8dac-d425bffac868" path="/var/lib/kubelet/pods/be32b4f1-dd72-4485-8dac-d425bffac868/volumes" Sep 08 10:24:26.992635 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:26.992600 2561 generic.go:356] "Generic (PLEG): container finished" podID="b4df702c-8acd-49f0-811a-dba71704df24" containerID="ff56ee8b94923bea52fbd4916ca309ae17e134cfeedc69bdf12386c77c9c02a2" exitCode=0 Sep 08 10:24:26.993028 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:26.992680 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" event={"ID":"b4df702c-8acd-49f0-811a-dba71704df24","Type":"ContainerDied","Data":"ff56ee8b94923bea52fbd4916ca309ae17e134cfeedc69bdf12386c77c9c02a2"} Sep 08 10:24:27.999635 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:27.999599 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" event={"ID":"b4df702c-8acd-49f0-811a-dba71704df24","Type":"ContainerStarted","Data":"b4cae5378a4c1a7f8867d3aa0b5addd4b1112c929241937309def3e4411e362e"} Sep 08 10:24:34.027910 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:34.027872 2561 generic.go:356] "Generic (PLEG): container finished" podID="b4df702c-8acd-49f0-811a-dba71704df24" containerID="b4cae5378a4c1a7f8867d3aa0b5addd4b1112c929241937309def3e4411e362e" exitCode=0 Sep 08 10:24:34.028371 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:34.027946 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" event={"ID":"b4df702c-8acd-49f0-811a-dba71704df24","Type":"ContainerDied","Data":"b4cae5378a4c1a7f8867d3aa0b5addd4b1112c929241937309def3e4411e362e"} Sep 08 10:24:34.041063 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:34.041040 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 08 10:24:35.038295 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:35.038250 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" event={"ID":"b4df702c-8acd-49f0-811a-dba71704df24","Type":"ContainerStarted","Data":"96445ed226f8ff8acc9bf293bb71f40e17060f8f324d4d9cda596dfc09a42ac2"} Sep 08 10:24:35.038826 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:35.038331 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" event={"ID":"b4df702c-8acd-49f0-811a-dba71704df24","Type":"ContainerStarted","Data":"9c34104a5d2ad5b0c7cfaa11b41346b3a58e13860f470d1586ab9d40d6be4dc2"} Sep 08 10:24:35.038826 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:35.038357 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" event={"ID":"b4df702c-8acd-49f0-811a-dba71704df24","Type":"ContainerStarted","Data":"33bc4a4bff3d1b3176d9125d6fbee0db61faea062a3f1cb8bc86b3e2c0d0808e"} Sep 08 10:24:35.038826 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:35.038384 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" event={"ID":"b4df702c-8acd-49f0-811a-dba71704df24","Type":"ContainerStarted","Data":"b27b3c21a9403e2a9db97c8b059c087aa826c181599b140b0b7a1a6f71aaa2d5"} Sep 08 10:24:35.081548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:35.081464 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podStartSLOduration=15.52117954 podStartE2EDuration="42.081438671s" totalImagesPullingTime="26.560259131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:23:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:24:35.079034411 +0000 UTC m=+451.448299709" watchObservedRunningTime="2026-09-08 10:24:35.081438671 +0000 UTC m=+451.450703968" Sep 08 10:24:36.830750 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:36.830691 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:24:36.831375 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:36.831337 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:24:36.831375 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:36.831373 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:24:36.831570 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:36.831392 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:24:36.833023 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:36.832975 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 08 10:24:36.833023 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:36.833009 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.27:8816: connect: connection refused" Sep 08 10:24:36.833198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:36.833110 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.27:6571: connect: connection refused" Sep 08 10:24:36.833301 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:36.833011 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Sep 08 10:24:39.831499 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:39.831451 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.27:6571: connect: connection refused" Sep 08 10:24:39.832011 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:39.831448 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 08 10:24:39.832011 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:39.831448 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.27:8816: connect: connection refused" Sep 08 10:24:39.832011 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:39.831464 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Sep 08 10:24:42.831244 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:42.831193 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 08 10:24:42.831809 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:42.831764 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:24:42.832093 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:42.832070 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:24:42.832433 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:42.832408 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:24:42.832583 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:42.832462 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:24:42.832656 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:42.832592 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:24:42.832711 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:42.832665 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:24:42.837559 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:42.837516 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:24:43.087983 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:43.087907 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:24:43.091040 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:43.091018 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:24:44.831179 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:44.831150 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:24:45.830787 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:45.830754 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:24:46.101308 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:24:46.101230 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:25:48.240351 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.240311 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4"] Sep 08 10:25:48.240959 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.240619 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="registry" containerID="cri-o://b27b3c21a9403e2a9db97c8b059c087aa826c181599b140b0b7a1a6f71aaa2d5" gracePeriod=30 Sep 08 10:25:48.240959 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.240837 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="ui" containerID="cri-o://96445ed226f8ff8acc9bf293bb71f40e17060f8f324d4d9cda596dfc09a42ac2" gracePeriod=30 Sep 08 10:25:48.240959 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.240932 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="offline" containerID="cri-o://9c34104a5d2ad5b0c7cfaa11b41346b3a58e13860f470d1586ab9d40d6be4dc2" gracePeriod=30 Sep 08 10:25:48.241128 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.241003 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="online" containerID="cri-o://33bc4a4bff3d1b3176d9125d6fbee0db61faea062a3f1cb8bc86b3e2c0d0808e" gracePeriod=30 Sep 08 10:25:48.676786 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.676676 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw"] Sep 08 10:25:48.677080 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.677060 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be32b4f1-dd72-4485-8dac-d425bffac868" containerName="console" Sep 08 10:25:48.677080 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.677081 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="be32b4f1-dd72-4485-8dac-d425bffac868" containerName="console" Sep 08 10:25:48.677225 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.677160 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="be32b4f1-dd72-4485-8dac-d425bffac868" containerName="console" Sep 08 10:25:48.681042 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.681008 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.683802 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.683772 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-z9lkr\"" Sep 08 10:25:48.693857 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.693824 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw"] Sep 08 10:25:48.720887 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.720841 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-online-tls\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.721070 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.720904 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-offline-tls\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.721070 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.720985 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0982ad91-531a-4b81-be40-bf428b7509b4-feast-data\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.721070 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.721029 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-registry-tls\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.721171 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.721116 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-ui-tls\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.721204 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.721170 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmflq\" (UniqueName: \"kubernetes.io/projected/0982ad91-531a-4b81-be40-bf428b7509b4-kube-api-access-dmflq\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.822436 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.822400 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0982ad91-531a-4b81-be40-bf428b7509b4-feast-data\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.822660 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.822454 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-registry-tls\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.822660 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.822507 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-ui-tls\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.822660 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.822560 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmflq\" (UniqueName: \"kubernetes.io/projected/0982ad91-531a-4b81-be40-bf428b7509b4-kube-api-access-dmflq\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.822660 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.822642 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-online-tls\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.822882 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.822669 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-offline-tls\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.822956 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.822920 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0982ad91-531a-4b81-be40-bf428b7509b4-feast-data\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.823010 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:25:48.822983 2561 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 08 10:25:48.823058 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:25:48.823052 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-online-tls podName:0982ad91-531a-4b81-be40-bf428b7509b4 nodeName:}" failed. No retries permitted until 2026-09-08 10:25:49.323030415 +0000 UTC m=+525.692295694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-online-tls") pod "feast-simple-feast-setup-8456cc85bf-x62sw" (UID: "0982ad91-531a-4b81-be40-bf428b7509b4") : secret "feast-simple-feast-setup-online-tls" not found Sep 08 10:25:48.825339 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.825308 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-offline-tls\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.825482 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.825349 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-registry-tls\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.825482 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.825438 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-ui-tls\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:48.834156 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:48.834110 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmflq\" (UniqueName: \"kubernetes.io/projected/0982ad91-531a-4b81-be40-bf428b7509b4-kube-api-access-dmflq\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:49.329134 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:49.329084 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-online-tls\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:49.331695 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:49.331664 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-online-tls\") pod \"feast-simple-feast-setup-8456cc85bf-x62sw\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:49.366927 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:49.366886 2561 generic.go:356] "Generic (PLEG): container finished" podID="b4df702c-8acd-49f0-811a-dba71704df24" containerID="96445ed226f8ff8acc9bf293bb71f40e17060f8f324d4d9cda596dfc09a42ac2" exitCode=0 Sep 08 10:25:49.366927 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:49.366919 2561 generic.go:356] "Generic (PLEG): container finished" podID="b4df702c-8acd-49f0-811a-dba71704df24" containerID="33bc4a4bff3d1b3176d9125d6fbee0db61faea062a3f1cb8bc86b3e2c0d0808e" exitCode=0 Sep 08 10:25:49.367139 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:49.366939 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" event={"ID":"b4df702c-8acd-49f0-811a-dba71704df24","Type":"ContainerDied","Data":"96445ed226f8ff8acc9bf293bb71f40e17060f8f324d4d9cda596dfc09a42ac2"} Sep 08 10:25:49.367139 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:49.366985 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" event={"ID":"b4df702c-8acd-49f0-811a-dba71704df24","Type":"ContainerDied","Data":"33bc4a4bff3d1b3176d9125d6fbee0db61faea062a3f1cb8bc86b3e2c0d0808e"} Sep 08 10:25:49.592574 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:49.592476 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:25:49.878373 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:49.878343 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw"] Sep 08 10:25:49.879744 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:25:49.879707 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0982ad91_531a_4b81_be40_bf428b7509b4.slice/crio-349e875ff93605b9bcd5e674afdf7e86734867f86d0332d1c08336544f113594 WatchSource:0}: Error finding container 349e875ff93605b9bcd5e674afdf7e86734867f86d0332d1c08336544f113594: Status 404 returned error can't find the container with id 349e875ff93605b9bcd5e674afdf7e86734867f86d0332d1c08336544f113594 Sep 08 10:25:50.373681 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:50.373637 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" event={"ID":"0982ad91-531a-4b81-be40-bf428b7509b4","Type":"ContainerStarted","Data":"19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e"} Sep 08 10:25:50.374078 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:50.373694 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" event={"ID":"0982ad91-531a-4b81-be40-bf428b7509b4","Type":"ContainerStarted","Data":"349e875ff93605b9bcd5e674afdf7e86734867f86d0332d1c08336544f113594"} Sep 08 10:25:53.089146 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:53.089098 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Sep 08 10:25:55.399697 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:55.399660 2561 generic.go:356] "Generic (PLEG): container finished" podID="0982ad91-531a-4b81-be40-bf428b7509b4" containerID="19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e" exitCode=0 Sep 08 10:25:55.400065 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:55.399732 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" event={"ID":"0982ad91-531a-4b81-be40-bf428b7509b4","Type":"ContainerDied","Data":"19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e"} Sep 08 10:25:56.101412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:56.101367 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 08 10:25:56.405812 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:25:56.405711 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" event={"ID":"0982ad91-531a-4b81-be40-bf428b7509b4","Type":"ContainerStarted","Data":"2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5"} Sep 08 10:26:01.429045 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:01.428962 2561 generic.go:356] "Generic (PLEG): container finished" podID="0982ad91-531a-4b81-be40-bf428b7509b4" containerID="2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5" exitCode=0 Sep 08 10:26:01.429045 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:01.429001 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" event={"ID":"0982ad91-531a-4b81-be40-bf428b7509b4","Type":"ContainerDied","Data":"2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5"} Sep 08 10:26:02.440298 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:02.440251 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" event={"ID":"0982ad91-531a-4b81-be40-bf428b7509b4","Type":"ContainerStarted","Data":"31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d"} Sep 08 10:26:02.440802 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:02.440319 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" event={"ID":"0982ad91-531a-4b81-be40-bf428b7509b4","Type":"ContainerStarted","Data":"1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c"} Sep 08 10:26:02.440802 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:02.440343 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" event={"ID":"0982ad91-531a-4b81-be40-bf428b7509b4","Type":"ContainerStarted","Data":"414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9"} Sep 08 10:26:02.440802 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:02.440360 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" event={"ID":"0982ad91-531a-4b81-be40-bf428b7509b4","Type":"ContainerStarted","Data":"4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125"} Sep 08 10:26:02.470754 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:02.470686 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podStartSLOduration=14.470665163 podStartE2EDuration="14.470665163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:25:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:26:02.466617821 +0000 UTC m=+538.835883121" watchObservedRunningTime="2026-09-08 10:26:02.470665163 +0000 UTC m=+538.839930461" Sep 08 10:26:03.088628 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:03.088569 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Sep 08 10:26:04.592946 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:04.592908 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:26:04.593391 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:04.592966 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:26:04.593391 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:04.592987 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:26:04.593391 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:04.593004 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:26:04.595036 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:04.594979 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 08 10:26:04.595036 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:04.595028 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.28:8816: connect: connection refused" Sep 08 10:26:04.595230 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:04.594987 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.28:6571: connect: connection refused" Sep 08 10:26:04.595230 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:04.594991 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 08 10:26:06.101355 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:06.101302 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 08 10:26:07.593558 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:07.593489 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 08 10:26:07.594105 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:07.593509 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 08 10:26:07.594250 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:07.593496 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.28:6571: connect: connection refused" Sep 08 10:26:07.594355 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:07.593551 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.28:8816: connect: connection refused" Sep 08 10:26:09.592704 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:09.592661 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:26:09.593057 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:09.592725 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:26:09.593057 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:09.592741 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:26:09.593057 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:09.592757 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:26:10.593946 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:10.593913 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:26:10.594493 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:10.594464 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:26:10.594678 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:10.594662 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:26:10.594737 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:10.594727 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:26:10.599378 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:10.599359 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:26:11.492465 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:11.492435 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:26:11.492646 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:11.492503 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:26:11.495061 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:11.495042 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:26:13.088798 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:13.088757 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Sep 08 10:26:13.089168 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:13.088863 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:26:16.101258 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:16.101218 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Sep 08 10:26:16.101737 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:16.101333 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:26:18.521880 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.521802 2561 generic.go:356] "Generic (PLEG): container finished" podID="b4df702c-8acd-49f0-811a-dba71704df24" containerID="9c34104a5d2ad5b0c7cfaa11b41346b3a58e13860f470d1586ab9d40d6be4dc2" exitCode=137 Sep 08 10:26:18.521880 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.521827 2561 generic.go:356] "Generic (PLEG): container finished" podID="b4df702c-8acd-49f0-811a-dba71704df24" containerID="b27b3c21a9403e2a9db97c8b059c087aa826c181599b140b0b7a1a6f71aaa2d5" exitCode=137 Sep 08 10:26:18.522219 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.521872 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" event={"ID":"b4df702c-8acd-49f0-811a-dba71704df24","Type":"ContainerDied","Data":"9c34104a5d2ad5b0c7cfaa11b41346b3a58e13860f470d1586ab9d40d6be4dc2"} Sep 08 10:26:18.522219 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.521914 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" event={"ID":"b4df702c-8acd-49f0-811a-dba71704df24","Type":"ContainerDied","Data":"b27b3c21a9403e2a9db97c8b059c087aa826c181599b140b0b7a1a6f71aaa2d5"} Sep 08 10:26:18.877537 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.877496 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:26:18.987377 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.987350 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-offline-tls\") pod \"b4df702c-8acd-49f0-811a-dba71704df24\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " Sep 08 10:26:18.987488 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.987387 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-ui-tls\") pod \"b4df702c-8acd-49f0-811a-dba71704df24\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " Sep 08 10:26:18.987488 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.987416 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qll4p\" (UniqueName: \"kubernetes.io/projected/b4df702c-8acd-49f0-811a-dba71704df24-kube-api-access-qll4p\") pod \"b4df702c-8acd-49f0-811a-dba71704df24\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " Sep 08 10:26:18.987488 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.987438 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-online-tls\") pod \"b4df702c-8acd-49f0-811a-dba71704df24\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " Sep 08 10:26:18.987488 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.987455 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-registry-tls\") pod \"b4df702c-8acd-49f0-811a-dba71704df24\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " Sep 08 10:26:18.987488 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.987476 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b4df702c-8acd-49f0-811a-dba71704df24-feast-data\") pod \"b4df702c-8acd-49f0-811a-dba71704df24\" (UID: \"b4df702c-8acd-49f0-811a-dba71704df24\") " Sep 08 10:26:18.988206 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.988158 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4df702c-8acd-49f0-811a-dba71704df24-feast-data" (OuterVolumeSpecName: "feast-data") pod "b4df702c-8acd-49f0-811a-dba71704df24" (UID: "b4df702c-8acd-49f0-811a-dba71704df24"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:26:18.989690 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.989661 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-online-tls" (OuterVolumeSpecName: "online-tls") pod "b4df702c-8acd-49f0-811a-dba71704df24" (UID: "b4df702c-8acd-49f0-811a-dba71704df24"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:26:18.989788 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.989684 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "b4df702c-8acd-49f0-811a-dba71704df24" (UID: "b4df702c-8acd-49f0-811a-dba71704df24"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:26:18.989788 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.989705 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b4df702c-8acd-49f0-811a-dba71704df24" (UID: "b4df702c-8acd-49f0-811a-dba71704df24"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:26:18.989875 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.989859 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4df702c-8acd-49f0-811a-dba71704df24-kube-api-access-qll4p" (OuterVolumeSpecName: "kube-api-access-qll4p") pod "b4df702c-8acd-49f0-811a-dba71704df24" (UID: "b4df702c-8acd-49f0-811a-dba71704df24"). InnerVolumeSpecName "kube-api-access-qll4p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:26:18.989939 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:18.989927 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "b4df702c-8acd-49f0-811a-dba71704df24" (UID: "b4df702c-8acd-49f0-811a-dba71704df24"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:26:19.088263 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:19.088200 2561 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b4df702c-8acd-49f0-811a-dba71704df24-feast-data\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:26:19.088263 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:19.088225 2561 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-offline-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:26:19.088263 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:19.088233 2561 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-ui-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:26:19.088263 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:19.088243 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qll4p\" (UniqueName: \"kubernetes.io/projected/b4df702c-8acd-49f0-811a-dba71704df24-kube-api-access-qll4p\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:26:19.088467 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:19.088271 2561 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-online-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:26:19.088467 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:19.088281 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b4df702c-8acd-49f0-811a-dba71704df24-registry-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:26:19.529087 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:19.529057 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" event={"ID":"b4df702c-8acd-49f0-811a-dba71704df24","Type":"ContainerDied","Data":"02f4a46265f6662e9ab0318513768b30498c6fdafaaf2dd7e66c4cb4c76b95c4"} Sep 08 10:26:19.529458 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:19.529102 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4" Sep 08 10:26:19.529458 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:19.529114 2561 scope.go:117] "RemoveContainer" containerID="96445ed226f8ff8acc9bf293bb71f40e17060f8f324d4d9cda596dfc09a42ac2" Sep 08 10:26:19.539750 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:19.539734 2561 scope.go:117] "RemoveContainer" containerID="9c34104a5d2ad5b0c7cfaa11b41346b3a58e13860f470d1586ab9d40d6be4dc2" Sep 08 10:26:19.547298 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:19.547281 2561 scope.go:117] "RemoveContainer" containerID="33bc4a4bff3d1b3176d9125d6fbee0db61faea062a3f1cb8bc86b3e2c0d0808e" Sep 08 10:26:19.554106 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:19.554088 2561 scope.go:117] "RemoveContainer" containerID="b27b3c21a9403e2a9db97c8b059c087aa826c181599b140b0b7a1a6f71aaa2d5" Sep 08 10:26:19.562250 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:19.562227 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4"] Sep 08 10:26:19.562630 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:19.562614 2561 scope.go:117] "RemoveContainer" containerID="b4cae5378a4c1a7f8867d3aa0b5addd4b1112c929241937309def3e4411e362e" Sep 08 10:26:19.564173 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:19.564153 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8456cc85bf-t5zc4"] Sep 08 10:26:19.569605 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:19.569591 2561 scope.go:117] "RemoveContainer" containerID="ff56ee8b94923bea52fbd4916ca309ae17e134cfeedc69bdf12386c77c9c02a2" Sep 08 10:26:20.145920 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:26:20.145891 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4df702c-8acd-49f0-811a-dba71704df24" path="/var/lib/kubelet/pods/b4df702c-8acd-49f0-811a-dba71704df24/volumes" Sep 08 10:27:13.948828 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.948784 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9"] Sep 08 10:27:13.949412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.949207 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="online" Sep 08 10:27:13.949412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.949221 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="online" Sep 08 10:27:13.949412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.949233 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="offline" Sep 08 10:27:13.949412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.949239 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="offline" Sep 08 10:27:13.949412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.949257 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="ui" Sep 08 10:27:13.949412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.949263 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="ui" Sep 08 10:27:13.949412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.949274 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="feast-apply" Sep 08 10:27:13.949412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.949280 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="feast-apply" Sep 08 10:27:13.949412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.949289 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="feast-init" Sep 08 10:27:13.949412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.949295 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="feast-init" Sep 08 10:27:13.949412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.949301 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="registry" Sep 08 10:27:13.949412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.949307 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="registry" Sep 08 10:27:13.949412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.949366 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="online" Sep 08 10:27:13.949412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.949375 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="offline" Sep 08 10:27:13.949412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.949383 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="registry" Sep 08 10:27:13.949412 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.949390 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4df702c-8acd-49f0-811a-dba71704df24" containerName="ui" Sep 08 10:27:13.954422 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.954395 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:13.957172 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.957133 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 08 10:27:13.957348 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.957133 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 08 10:27:13.957348 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.957135 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 08 10:27:13.957348 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.957144 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-rkb29\"" Sep 08 10:27:13.957348 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.957147 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 08 10:27:13.957580 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.957175 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 08 10:27:13.957636 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.957596 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 08 10:27:13.957691 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.957651 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 08 10:27:13.966155 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:13.966123 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9"] Sep 08 10:27:14.039164 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.039122 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkp9t\" (UniqueName: \"kubernetes.io/projected/0d58b666-84fe-4c1b-9235-778978e26097-kube-api-access-pkp9t\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.039164 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.039174 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-offline-tls\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.039365 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.039196 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0d58b666-84fe-4c1b-9235-778978e26097-registry-tls\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.039365 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.039326 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-ui-tls\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.039365 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.039359 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-online-tls\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.039479 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.039391 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0d58b666-84fe-4c1b-9235-778978e26097-feast-data\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.140099 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.140064 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0d58b666-84fe-4c1b-9235-778978e26097-registry-tls\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.140253 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.140185 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-ui-tls\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.140253 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.140211 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-online-tls\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.140366 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.140253 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0d58b666-84fe-4c1b-9235-778978e26097-feast-data\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.140366 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.140281 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkp9t\" (UniqueName: \"kubernetes.io/projected/0d58b666-84fe-4c1b-9235-778978e26097-kube-api-access-pkp9t\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.140366 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.140312 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-offline-tls\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.140366 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:27:14.140344 2561 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 08 10:27:14.140714 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:27:14.140435 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-ui-tls podName:0d58b666-84fe-4c1b-9235-778978e26097 nodeName:}" failed. No retries permitted until 2026-09-08 10:27:14.640412786 +0000 UTC m=+611.009678074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-ui-tls") pod "feast-simple-feast-remote-setup-77c886f7c6-lrhp9" (UID: "0d58b666-84fe-4c1b-9235-778978e26097") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 08 10:27:14.141047 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.141001 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0d58b666-84fe-4c1b-9235-778978e26097-feast-data\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.141316 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.141293 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0d58b666-84fe-4c1b-9235-778978e26097-registry-tls\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.149693 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.149668 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-online-tls\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.151273 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.151252 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-offline-tls\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.154077 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.154048 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkp9t\" (UniqueName: \"kubernetes.io/projected/0d58b666-84fe-4c1b-9235-778978e26097-kube-api-access-pkp9t\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.645393 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.645346 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-ui-tls\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.647873 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.647837 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-ui-tls\") pod \"feast-simple-feast-remote-setup-77c886f7c6-lrhp9\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:14.865844 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:14.865790 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:15.148848 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:15.148815 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9"] Sep 08 10:27:15.150467 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:27:15.150421 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d58b666_84fe_4c1b_9235_778978e26097.slice/crio-827885cd046f16a85d6ab29e3a9d794943890417b50e1473d64cb064e0b2b64e WatchSource:0}: Error finding container 827885cd046f16a85d6ab29e3a9d794943890417b50e1473d64cb064e0b2b64e: Status 404 returned error can't find the container with id 827885cd046f16a85d6ab29e3a9d794943890417b50e1473d64cb064e0b2b64e Sep 08 10:27:15.774014 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:15.773976 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" event={"ID":"0d58b666-84fe-4c1b-9235-778978e26097","Type":"ContainerStarted","Data":"0de846c3b152c490918fafd9d9d5a6546c030fd74370a43a816c3625336cf12f"} Sep 08 10:27:15.774212 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:15.774031 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" event={"ID":"0d58b666-84fe-4c1b-9235-778978e26097","Type":"ContainerStarted","Data":"827885cd046f16a85d6ab29e3a9d794943890417b50e1473d64cb064e0b2b64e"} Sep 08 10:27:20.798243 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:20.798205 2561 generic.go:356] "Generic (PLEG): container finished" podID="0d58b666-84fe-4c1b-9235-778978e26097" containerID="0de846c3b152c490918fafd9d9d5a6546c030fd74370a43a816c3625336cf12f" exitCode=0 Sep 08 10:27:20.798711 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:20.798270 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" event={"ID":"0d58b666-84fe-4c1b-9235-778978e26097","Type":"ContainerDied","Data":"0de846c3b152c490918fafd9d9d5a6546c030fd74370a43a816c3625336cf12f"} Sep 08 10:27:21.805201 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:21.805167 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" event={"ID":"0d58b666-84fe-4c1b-9235-778978e26097","Type":"ContainerStarted","Data":"d51bb4b3a51758a447160388a241af894f9564edeaebf48955b5ab1e0c7b5bec"} Sep 08 10:27:47.923468 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:47.923432 2561 generic.go:356] "Generic (PLEG): container finished" podID="0d58b666-84fe-4c1b-9235-778978e26097" containerID="d51bb4b3a51758a447160388a241af894f9564edeaebf48955b5ab1e0c7b5bec" exitCode=0 Sep 08 10:27:47.923900 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:47.923503 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" event={"ID":"0d58b666-84fe-4c1b-9235-778978e26097","Type":"ContainerDied","Data":"d51bb4b3a51758a447160388a241af894f9564edeaebf48955b5ab1e0c7b5bec"} Sep 08 10:27:48.932581 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:48.932509 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" event={"ID":"0d58b666-84fe-4c1b-9235-778978e26097","Type":"ContainerStarted","Data":"9c29b3ae9e87b1324fb2e84b7f5368060e7a48070f685e05f75179d07be70693"} Sep 08 10:27:48.932983 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:48.932606 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" event={"ID":"0d58b666-84fe-4c1b-9235-778978e26097","Type":"ContainerStarted","Data":"8e2511fe562cbf240d2b75883fb0dd17d4d8d1bb78d5d8e09d50c784cddf1791"} Sep 08 10:27:48.932983 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:48.932626 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" event={"ID":"0d58b666-84fe-4c1b-9235-778978e26097","Type":"ContainerStarted","Data":"eadde550f49327dd8b87f04588653247aeb5f5fba42d8ba8c19af5f0ac36afde"} Sep 08 10:27:48.958688 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:48.958613 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" podStartSLOduration=35.958586496 podStartE2EDuration="35.958586496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:27:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:27:48.955945092 +0000 UTC m=+645.325210390" watchObservedRunningTime="2026-09-08 10:27:48.958586496 +0000 UTC m=+645.327851794" Sep 08 10:27:50.866049 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:50.866002 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:50.866577 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:50.866071 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:50.866577 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:50.866090 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:50.868201 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:50.868160 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Sep 08 10:27:50.868201 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:50.868179 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.29:8816: connect: connection refused" Sep 08 10:27:50.868373 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:50.868183 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Sep 08 10:27:53.867119 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:53.867071 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.29:8816: connect: connection refused" Sep 08 10:27:53.867593 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:53.867111 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Sep 08 10:27:53.867593 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:53.867072 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Sep 08 10:27:54.866585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:54.866509 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:54.866808 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:54.866609 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:54.866808 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:54.866638 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:56.867648 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:56.867610 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:56.868091 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:56.867743 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:56.868432 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:56.868407 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:56.868829 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:56.868809 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:56.890020 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:56.889993 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:27:56.999656 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:27:56.999623 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:28:58.253829 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:28:58.253755 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9"] Sep 08 10:28:58.254282 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:28:58.254039 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="online" containerID="cri-o://eadde550f49327dd8b87f04588653247aeb5f5fba42d8ba8c19af5f0ac36afde" gracePeriod=30 Sep 08 10:28:58.254282 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:28:58.254093 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="offline" containerID="cri-o://8e2511fe562cbf240d2b75883fb0dd17d4d8d1bb78d5d8e09d50c784cddf1791" gracePeriod=30 Sep 08 10:28:58.254282 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:28:58.254100 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="ui" containerID="cri-o://9c29b3ae9e87b1324fb2e84b7f5368060e7a48070f685e05f75179d07be70693" gracePeriod=30 Sep 08 10:28:58.353394 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:28:58.353360 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw"] Sep 08 10:28:58.353872 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:28:58.353685 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="ui" containerID="cri-o://31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d" gracePeriod=30 Sep 08 10:28:58.353872 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:28:58.353678 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="registry" containerID="cri-o://4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125" gracePeriod=30 Sep 08 10:28:58.353872 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:28:58.353734 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="online" containerID="cri-o://414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9" gracePeriod=30 Sep 08 10:28:58.353872 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:28:58.353730 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="offline" containerID="cri-o://1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c" gracePeriod=30 Sep 08 10:28:59.253988 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:28:59.253951 2561 generic.go:356] "Generic (PLEG): container finished" podID="0982ad91-531a-4b81-be40-bf428b7509b4" containerID="31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d" exitCode=0 Sep 08 10:28:59.254414 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:28:59.254024 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" event={"ID":"0982ad91-531a-4b81-be40-bf428b7509b4","Type":"ContainerDied","Data":"31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d"} Sep 08 10:28:59.256808 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:28:59.256784 2561 generic.go:356] "Generic (PLEG): container finished" podID="0d58b666-84fe-4c1b-9235-778978e26097" containerID="9c29b3ae9e87b1324fb2e84b7f5368060e7a48070f685e05f75179d07be70693" exitCode=0 Sep 08 10:28:59.256808 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:28:59.256802 2561 generic.go:356] "Generic (PLEG): container finished" podID="0d58b666-84fe-4c1b-9235-778978e26097" containerID="eadde550f49327dd8b87f04588653247aeb5f5fba42d8ba8c19af5f0ac36afde" exitCode=0 Sep 08 10:28:59.256968 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:28:59.256834 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" event={"ID":"0d58b666-84fe-4c1b-9235-778978e26097","Type":"ContainerDied","Data":"9c29b3ae9e87b1324fb2e84b7f5368060e7a48070f685e05f75179d07be70693"} Sep 08 10:28:59.256968 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:28:59.256859 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" event={"ID":"0d58b666-84fe-4c1b-9235-778978e26097","Type":"ContainerDied","Data":"eadde550f49327dd8b87f04588653247aeb5f5fba42d8ba8c19af5f0ac36afde"} Sep 08 10:29:00.264220 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:00.264176 2561 generic.go:356] "Generic (PLEG): container finished" podID="0982ad91-531a-4b81-be40-bf428b7509b4" containerID="414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9" exitCode=0 Sep 08 10:29:00.264627 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:00.264244 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" event={"ID":"0982ad91-531a-4b81-be40-bf428b7509b4","Type":"ContainerDied","Data":"414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9"} Sep 08 10:29:01.492756 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:01.492709 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 08 10:29:01.493168 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:01.492706 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 08 10:29:06.868401 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:06.868338 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Sep 08 10:29:06.978226 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:06.978187 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Sep 08 10:29:11.492793 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:11.492756 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 08 10:29:11.492793 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:11.492776 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 08 10:29:16.869234 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:16.869181 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Sep 08 10:29:16.978038 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:16.977986 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Sep 08 10:29:21.492661 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:21.492600 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Sep 08 10:29:21.492661 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:21.492627 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Sep 08 10:29:21.493119 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:21.492733 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:29:21.493119 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:21.492790 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:29:26.868476 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:26.868413 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Sep 08 10:29:26.868914 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:26.868611 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:29:26.978124 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:26.978088 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Sep 08 10:29:26.978257 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:26.978190 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:29:28.387342 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:28.387306 2561 generic.go:356] "Generic (PLEG): container finished" podID="0d58b666-84fe-4c1b-9235-778978e26097" containerID="8e2511fe562cbf240d2b75883fb0dd17d4d8d1bb78d5d8e09d50c784cddf1791" exitCode=137 Sep 08 10:29:28.387663 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:28.387375 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" event={"ID":"0d58b666-84fe-4c1b-9235-778978e26097","Type":"ContainerDied","Data":"8e2511fe562cbf240d2b75883fb0dd17d4d8d1bb78d5d8e09d50c784cddf1791"} Sep 08 10:29:28.903182 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:28.903153 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:29:28.992783 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:28.992759 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:29:29.003704 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.003683 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0d58b666-84fe-4c1b-9235-778978e26097-feast-data\") pod \"0d58b666-84fe-4c1b-9235-778978e26097\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " Sep 08 10:29:29.003769 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.003721 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0d58b666-84fe-4c1b-9235-778978e26097-registry-tls\") pod \"0d58b666-84fe-4c1b-9235-778978e26097\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " Sep 08 10:29:29.003769 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.003751 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-offline-tls\") pod \"0d58b666-84fe-4c1b-9235-778978e26097\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " Sep 08 10:29:29.003845 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.003781 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-online-tls\") pod \"0d58b666-84fe-4c1b-9235-778978e26097\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " Sep 08 10:29:29.003845 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.003827 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-ui-tls\") pod \"0d58b666-84fe-4c1b-9235-778978e26097\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " Sep 08 10:29:29.003936 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.003868 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkp9t\" (UniqueName: \"kubernetes.io/projected/0d58b666-84fe-4c1b-9235-778978e26097-kube-api-access-pkp9t\") pod \"0d58b666-84fe-4c1b-9235-778978e26097\" (UID: \"0d58b666-84fe-4c1b-9235-778978e26097\") " Sep 08 10:29:29.004142 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.004104 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d58b666-84fe-4c1b-9235-778978e26097-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0d58b666-84fe-4c1b-9235-778978e26097" (UID: "0d58b666-84fe-4c1b-9235-778978e26097"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 10:29:29.004241 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.004220 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d58b666-84fe-4c1b-9235-778978e26097-feast-data" (OuterVolumeSpecName: "feast-data") pod "0d58b666-84fe-4c1b-9235-778978e26097" (UID: "0d58b666-84fe-4c1b-9235-778978e26097"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:29:29.005893 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.005868 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "0d58b666-84fe-4c1b-9235-778978e26097" (UID: "0d58b666-84fe-4c1b-9235-778978e26097"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:29:29.005977 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.005905 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "0d58b666-84fe-4c1b-9235-778978e26097" (UID: "0d58b666-84fe-4c1b-9235-778978e26097"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:29:29.005977 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.005957 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-online-tls" (OuterVolumeSpecName: "online-tls") pod "0d58b666-84fe-4c1b-9235-778978e26097" (UID: "0d58b666-84fe-4c1b-9235-778978e26097"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:29:29.005977 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.005964 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d58b666-84fe-4c1b-9235-778978e26097-kube-api-access-pkp9t" (OuterVolumeSpecName: "kube-api-access-pkp9t") pod "0d58b666-84fe-4c1b-9235-778978e26097" (UID: "0d58b666-84fe-4c1b-9235-778978e26097"). InnerVolumeSpecName "kube-api-access-pkp9t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:29:29.104519 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.104459 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-registry-tls\") pod \"0982ad91-531a-4b81-be40-bf428b7509b4\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " Sep 08 10:29:29.104519 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.104488 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0982ad91-531a-4b81-be40-bf428b7509b4-feast-data\") pod \"0982ad91-531a-4b81-be40-bf428b7509b4\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " Sep 08 10:29:29.104519 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.104517 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-ui-tls\") pod \"0982ad91-531a-4b81-be40-bf428b7509b4\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " Sep 08 10:29:29.104761 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.104566 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-online-tls\") pod \"0982ad91-531a-4b81-be40-bf428b7509b4\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " Sep 08 10:29:29.104761 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.104607 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmflq\" (UniqueName: \"kubernetes.io/projected/0982ad91-531a-4b81-be40-bf428b7509b4-kube-api-access-dmflq\") pod \"0982ad91-531a-4b81-be40-bf428b7509b4\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " Sep 08 10:29:29.104761 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.104653 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-offline-tls\") pod \"0982ad91-531a-4b81-be40-bf428b7509b4\" (UID: \"0982ad91-531a-4b81-be40-bf428b7509b4\") " Sep 08 10:29:29.104911 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.104895 2561 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0d58b666-84fe-4c1b-9235-778978e26097-feast-data\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:29:29.104969 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.104912 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0d58b666-84fe-4c1b-9235-778978e26097-registry-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:29:29.104969 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.104924 2561 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-offline-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:29:29.104969 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.104936 2561 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-online-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:29:29.104969 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.104948 2561 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0d58b666-84fe-4c1b-9235-778978e26097-ui-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:29:29.104969 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.104960 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkp9t\" (UniqueName: \"kubernetes.io/projected/0d58b666-84fe-4c1b-9235-778978e26097-kube-api-access-pkp9t\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:29:29.105272 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.105159 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0982ad91-531a-4b81-be40-bf428b7509b4-feast-data" (OuterVolumeSpecName: "feast-data") pod "0982ad91-531a-4b81-be40-bf428b7509b4" (UID: "0982ad91-531a-4b81-be40-bf428b7509b4"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:29:29.106771 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.106717 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0982ad91-531a-4b81-be40-bf428b7509b4-kube-api-access-dmflq" (OuterVolumeSpecName: "kube-api-access-dmflq") pod "0982ad91-531a-4b81-be40-bf428b7509b4" (UID: "0982ad91-531a-4b81-be40-bf428b7509b4"). InnerVolumeSpecName "kube-api-access-dmflq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:29:29.106868 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.106833 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0982ad91-531a-4b81-be40-bf428b7509b4" (UID: "0982ad91-531a-4b81-be40-bf428b7509b4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:29:29.106936 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.106913 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "0982ad91-531a-4b81-be40-bf428b7509b4" (UID: "0982ad91-531a-4b81-be40-bf428b7509b4"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:29:29.107129 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.107107 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-online-tls" (OuterVolumeSpecName: "online-tls") pod "0982ad91-531a-4b81-be40-bf428b7509b4" (UID: "0982ad91-531a-4b81-be40-bf428b7509b4"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:29:29.107214 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.107199 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "0982ad91-531a-4b81-be40-bf428b7509b4" (UID: "0982ad91-531a-4b81-be40-bf428b7509b4"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:29:29.205943 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.205920 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-registry-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:29:29.205943 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.205941 2561 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0982ad91-531a-4b81-be40-bf428b7509b4-feast-data\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:29:29.206072 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.205950 2561 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-ui-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:29:29.206072 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.205958 2561 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-online-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:29:29.206072 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.205967 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmflq\" (UniqueName: \"kubernetes.io/projected/0982ad91-531a-4b81-be40-bf428b7509b4-kube-api-access-dmflq\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:29:29.206072 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.205977 2561 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0982ad91-531a-4b81-be40-bf428b7509b4-offline-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:29:29.394794 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.394735 2561 generic.go:356] "Generic (PLEG): container finished" podID="0982ad91-531a-4b81-be40-bf428b7509b4" containerID="1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c" exitCode=137 Sep 08 10:29:29.394794 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.394759 2561 generic.go:356] "Generic (PLEG): container finished" podID="0982ad91-531a-4b81-be40-bf428b7509b4" containerID="4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125" exitCode=137 Sep 08 10:29:29.395119 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.394798 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" Sep 08 10:29:29.395119 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.394817 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" event={"ID":"0982ad91-531a-4b81-be40-bf428b7509b4","Type":"ContainerDied","Data":"1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c"} Sep 08 10:29:29.395119 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.394860 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" event={"ID":"0982ad91-531a-4b81-be40-bf428b7509b4","Type":"ContainerDied","Data":"4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125"} Sep 08 10:29:29.395119 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.394879 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw" event={"ID":"0982ad91-531a-4b81-be40-bf428b7509b4","Type":"ContainerDied","Data":"349e875ff93605b9bcd5e674afdf7e86734867f86d0332d1c08336544f113594"} Sep 08 10:29:29.395119 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.394905 2561 scope.go:117] "RemoveContainer" containerID="31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d" Sep 08 10:29:29.397940 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.397915 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" event={"ID":"0d58b666-84fe-4c1b-9235-778978e26097","Type":"ContainerDied","Data":"827885cd046f16a85d6ab29e3a9d794943890417b50e1473d64cb064e0b2b64e"} Sep 08 10:29:29.398048 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.397965 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9" Sep 08 10:29:29.405111 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.405091 2561 scope.go:117] "RemoveContainer" containerID="1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c" Sep 08 10:29:29.412902 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.412885 2561 scope.go:117] "RemoveContainer" containerID="414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9" Sep 08 10:29:29.420038 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.420017 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw"] Sep 08 10:29:29.420810 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.420792 2561 scope.go:117] "RemoveContainer" containerID="4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125" Sep 08 10:29:29.426291 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.426266 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8456cc85bf-x62sw"] Sep 08 10:29:29.428737 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.428719 2561 scope.go:117] "RemoveContainer" containerID="2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5" Sep 08 10:29:29.435439 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.435424 2561 scope.go:117] "RemoveContainer" containerID="19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e" Sep 08 10:29:29.438197 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.438177 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9"] Sep 08 10:29:29.442597 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.442576 2561 scope.go:117] "RemoveContainer" containerID="31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d" Sep 08 10:29:29.442888 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:29:29.442872 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d\": container with ID starting with 31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d not found: ID does not exist" containerID="31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d" Sep 08 10:29:29.442951 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.442895 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d"} err="failed to get container status \"31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d\": rpc error: code = NotFound desc = could not find container \"31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d\": container with ID starting with 31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d not found: ID does not exist" Sep 08 10:29:29.442951 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.442911 2561 scope.go:117] "RemoveContainer" containerID="1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c" Sep 08 10:29:29.443158 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:29:29.443142 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c\": container with ID starting with 1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c not found: ID does not exist" containerID="1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c" Sep 08 10:29:29.443199 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.443164 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c"} err="failed to get container status \"1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c\": rpc error: code = NotFound desc = could not find container \"1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c\": container with ID starting with 1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c not found: ID does not exist" Sep 08 10:29:29.443199 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.443178 2561 scope.go:117] "RemoveContainer" containerID="414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9" Sep 08 10:29:29.443443 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:29:29.443422 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9\": container with ID starting with 414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9 not found: ID does not exist" containerID="414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9" Sep 08 10:29:29.443521 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.443450 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9"} err="failed to get container status \"414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9\": rpc error: code = NotFound desc = could not find container \"414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9\": container with ID starting with 414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9 not found: ID does not exist" Sep 08 10:29:29.443521 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.443475 2561 scope.go:117] "RemoveContainer" containerID="4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125" Sep 08 10:29:29.443875 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:29:29.443854 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125\": container with ID starting with 4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125 not found: ID does not exist" containerID="4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125" Sep 08 10:29:29.443972 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.443882 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125"} err="failed to get container status \"4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125\": rpc error: code = NotFound desc = could not find container \"4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125\": container with ID starting with 4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125 not found: ID does not exist" Sep 08 10:29:29.443972 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.443905 2561 scope.go:117] "RemoveContainer" containerID="2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5" Sep 08 10:29:29.444377 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:29:29.444264 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5\": container with ID starting with 2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5 not found: ID does not exist" containerID="2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5" Sep 08 10:29:29.444377 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.444295 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5"} err="failed to get container status \"2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5\": rpc error: code = NotFound desc = could not find container \"2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5\": container with ID starting with 2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5 not found: ID does not exist" Sep 08 10:29:29.444377 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.444315 2561 scope.go:117] "RemoveContainer" containerID="19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e" Sep 08 10:29:29.444649 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:29:29.444613 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e\": container with ID starting with 19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e not found: ID does not exist" containerID="19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e" Sep 08 10:29:29.444718 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.444653 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e"} err="failed to get container status \"19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e\": rpc error: code = NotFound desc = could not find container \"19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e\": container with ID starting with 19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e not found: ID does not exist" Sep 08 10:29:29.444718 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.444667 2561 scope.go:117] "RemoveContainer" containerID="31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d" Sep 08 10:29:29.444920 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.444900 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d"} err="failed to get container status \"31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d\": rpc error: code = NotFound desc = could not find container \"31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d\": container with ID starting with 31d50a742e3649ba52cf51f54d9d307c617ff924f262a91ee25eac3ceae3d76d not found: ID does not exist" Sep 08 10:29:29.444920 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.444920 2561 scope.go:117] "RemoveContainer" containerID="1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c" Sep 08 10:29:29.445207 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.445185 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c"} err="failed to get container status \"1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c\": rpc error: code = NotFound desc = could not find container \"1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c\": container with ID starting with 1ed23666928f61b012481926ecef82b1797b7dd6638a666f86807a088de8235c not found: ID does not exist" Sep 08 10:29:29.445207 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.445206 2561 scope.go:117] "RemoveContainer" containerID="414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9" Sep 08 10:29:29.445425 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.445406 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9"} err="failed to get container status \"414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9\": rpc error: code = NotFound desc = could not find container \"414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9\": container with ID starting with 414fe260f6f7234ac89a27380b9f7f67aad5d53ea3d72df8857af310f6cc53a9 not found: ID does not exist" Sep 08 10:29:29.445501 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.445427 2561 scope.go:117] "RemoveContainer" containerID="4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125" Sep 08 10:29:29.445728 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.445710 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125"} err="failed to get container status \"4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125\": rpc error: code = NotFound desc = could not find container \"4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125\": container with ID starting with 4cc81bfbc8aeb25bb7229dcf716f3bafe3a13211ff27120859c716570fc69125 not found: ID does not exist" Sep 08 10:29:29.445787 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.445728 2561 scope.go:117] "RemoveContainer" containerID="2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5" Sep 08 10:29:29.445969 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.445950 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5"} err="failed to get container status \"2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5\": rpc error: code = NotFound desc = could not find container \"2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5\": container with ID starting with 2a2865792a9df507a572e38479ad78d39646a8ed5fe0f9de03ec371368eca4b5 not found: ID does not exist" Sep 08 10:29:29.446036 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.445973 2561 scope.go:117] "RemoveContainer" containerID="19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e" Sep 08 10:29:29.446159 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.446140 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-77c886f7c6-lrhp9"] Sep 08 10:29:29.446233 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.446184 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e"} err="failed to get container status \"19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e\": rpc error: code = NotFound desc = could not find container \"19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e\": container with ID starting with 19e0e982a35206d979e2f9a99e9a7a88aaf5b9bb903f862c97d2a1fbe8e61e2e not found: ID does not exist" Sep 08 10:29:29.446233 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.446220 2561 scope.go:117] "RemoveContainer" containerID="9c29b3ae9e87b1324fb2e84b7f5368060e7a48070f685e05f75179d07be70693" Sep 08 10:29:29.452940 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.452926 2561 scope.go:117] "RemoveContainer" containerID="8e2511fe562cbf240d2b75883fb0dd17d4d8d1bb78d5d8e09d50c784cddf1791" Sep 08 10:29:29.459854 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.459839 2561 scope.go:117] "RemoveContainer" containerID="eadde550f49327dd8b87f04588653247aeb5f5fba42d8ba8c19af5f0ac36afde" Sep 08 10:29:29.466706 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.466690 2561 scope.go:117] "RemoveContainer" containerID="d51bb4b3a51758a447160388a241af894f9564edeaebf48955b5ab1e0c7b5bec" Sep 08 10:29:29.473226 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:29.473211 2561 scope.go:117] "RemoveContainer" containerID="0de846c3b152c490918fafd9d9d5a6546c030fd74370a43a816c3625336cf12f" Sep 08 10:29:30.152281 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:30.146226 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" path="/var/lib/kubelet/pods/0982ad91-531a-4b81-be40-bf428b7509b4/volumes" Sep 08 10:29:30.152281 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:30.147225 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d58b666-84fe-4c1b-9235-778978e26097" path="/var/lib/kubelet/pods/0d58b666-84fe-4c1b-9235-778978e26097/volumes" Sep 08 10:29:41.935598 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.935568 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-f4hmw"] Sep 08 10:29:41.936051 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936033 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="registry" Sep 08 10:29:41.936121 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936056 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="registry" Sep 08 10:29:41.936121 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936069 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="online" Sep 08 10:29:41.936121 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936078 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="online" Sep 08 10:29:41.936121 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936094 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="online" Sep 08 10:29:41.936121 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936103 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="online" Sep 08 10:29:41.936121 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936121 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="feast-init" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936130 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="feast-init" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936141 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="feast-apply" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936151 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="feast-apply" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936161 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="feast-apply" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936169 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="feast-apply" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936180 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="offline" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936188 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="offline" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936204 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="ui" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936213 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="ui" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936224 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="ui" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936232 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="ui" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936249 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="offline" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936257 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="offline" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936270 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="feast-init" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936278 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="feast-init" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936356 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="ui" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936370 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="ui" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936378 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="offline" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936388 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="online" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936398 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="offline" Sep 08 10:29:41.936405 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936408 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d58b666-84fe-4c1b-9235-778978e26097" containerName="online" Sep 08 10:29:41.937355 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.936420 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0982ad91-531a-4b81-be40-bf428b7509b4" containerName="registry" Sep 08 10:29:41.941375 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.941356 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-f4hmw" Sep 08 10:29:41.943905 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.943882 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 08 10:29:41.944023 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.943914 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 08 10:29:41.944023 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.943925 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-pjtfr\"" Sep 08 10:29:41.944444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.944424 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 08 10:29:41.948064 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.948045 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-f4hmw"] Sep 08 10:29:41.962761 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.962735 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-xlthg"] Sep 08 10:29:41.966443 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.966420 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-xlthg" Sep 08 10:29:41.972683 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:41.972663 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-xlthg"] Sep 08 10:29:42.000050 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:42.000022 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/77e9bb5c-3149-447a-ad13-c8a4ab91cad7-postgresdata\") pod \"postgres-864846d4f5-f4hmw\" (UID: \"77e9bb5c-3149-447a-ad13-c8a4ab91cad7\") " pod="test-ns-feast/postgres-864846d4f5-f4hmw" Sep 08 10:29:42.000166 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:42.000065 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wlrg\" (UniqueName: \"kubernetes.io/projected/77e9bb5c-3149-447a-ad13-c8a4ab91cad7-kube-api-access-8wlrg\") pod \"postgres-864846d4f5-f4hmw\" (UID: \"77e9bb5c-3149-447a-ad13-c8a4ab91cad7\") " pod="test-ns-feast/postgres-864846d4f5-f4hmw" Sep 08 10:29:42.100789 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:42.100759 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/77e9bb5c-3149-447a-ad13-c8a4ab91cad7-postgresdata\") pod \"postgres-864846d4f5-f4hmw\" (UID: \"77e9bb5c-3149-447a-ad13-c8a4ab91cad7\") " pod="test-ns-feast/postgres-864846d4f5-f4hmw" Sep 08 10:29:42.100911 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:42.100798 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hf97\" (UniqueName: \"kubernetes.io/projected/c8637904-e3b4-4b76-8908-a8fe50655647-kube-api-access-6hf97\") pod \"redis-69bbd798b6-xlthg\" (UID: \"c8637904-e3b4-4b76-8908-a8fe50655647\") " pod="test-ns-feast/redis-69bbd798b6-xlthg" Sep 08 10:29:42.100911 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:42.100825 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wlrg\" (UniqueName: \"kubernetes.io/projected/77e9bb5c-3149-447a-ad13-c8a4ab91cad7-kube-api-access-8wlrg\") pod \"postgres-864846d4f5-f4hmw\" (UID: \"77e9bb5c-3149-447a-ad13-c8a4ab91cad7\") " pod="test-ns-feast/postgres-864846d4f5-f4hmw" Sep 08 10:29:42.101111 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:42.101092 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/77e9bb5c-3149-447a-ad13-c8a4ab91cad7-postgresdata\") pod \"postgres-864846d4f5-f4hmw\" (UID: \"77e9bb5c-3149-447a-ad13-c8a4ab91cad7\") " pod="test-ns-feast/postgres-864846d4f5-f4hmw" Sep 08 10:29:42.110130 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:42.110103 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wlrg\" (UniqueName: \"kubernetes.io/projected/77e9bb5c-3149-447a-ad13-c8a4ab91cad7-kube-api-access-8wlrg\") pod \"postgres-864846d4f5-f4hmw\" (UID: \"77e9bb5c-3149-447a-ad13-c8a4ab91cad7\") " pod="test-ns-feast/postgres-864846d4f5-f4hmw" Sep 08 10:29:42.201417 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:42.201356 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hf97\" (UniqueName: \"kubernetes.io/projected/c8637904-e3b4-4b76-8908-a8fe50655647-kube-api-access-6hf97\") pod \"redis-69bbd798b6-xlthg\" (UID: \"c8637904-e3b4-4b76-8908-a8fe50655647\") " pod="test-ns-feast/redis-69bbd798b6-xlthg" Sep 08 10:29:42.209960 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:42.209941 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hf97\" (UniqueName: \"kubernetes.io/projected/c8637904-e3b4-4b76-8908-a8fe50655647-kube-api-access-6hf97\") pod \"redis-69bbd798b6-xlthg\" (UID: \"c8637904-e3b4-4b76-8908-a8fe50655647\") " pod="test-ns-feast/redis-69bbd798b6-xlthg" Sep 08 10:29:42.256899 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:42.256879 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-f4hmw" Sep 08 10:29:42.276809 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:42.276790 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-xlthg" Sep 08 10:29:42.529377 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:42.529350 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-f4hmw"] Sep 08 10:29:42.535241 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:29:42.535214 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77e9bb5c_3149_447a_ad13_c8a4ab91cad7.slice/crio-aee7b2a09100af4faa7c1f2c9a49149a28a37daebd2b060a4f93b5603e24e69e WatchSource:0}: Error finding container aee7b2a09100af4faa7c1f2c9a49149a28a37daebd2b060a4f93b5603e24e69e: Status 404 returned error can't find the container with id aee7b2a09100af4faa7c1f2c9a49149a28a37daebd2b060a4f93b5603e24e69e Sep 08 10:29:42.537073 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:42.537032 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 10:29:42.552804 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:42.552746 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-xlthg"] Sep 08 10:29:42.554622 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:29:42.554592 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8637904_e3b4_4b76_8908_a8fe50655647.slice/crio-9a0dbf95a8ef249394d45d652ccbcb6ed2945b8256a9cb83479c310b79ce7c8c WatchSource:0}: Error finding container 9a0dbf95a8ef249394d45d652ccbcb6ed2945b8256a9cb83479c310b79ce7c8c: Status 404 returned error can't find the container with id 9a0dbf95a8ef249394d45d652ccbcb6ed2945b8256a9cb83479c310b79ce7c8c Sep 08 10:29:43.458024 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:43.457983 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-xlthg" event={"ID":"c8637904-e3b4-4b76-8908-a8fe50655647","Type":"ContainerStarted","Data":"9a0dbf95a8ef249394d45d652ccbcb6ed2945b8256a9cb83479c310b79ce7c8c"} Sep 08 10:29:43.459332 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:43.459297 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-f4hmw" event={"ID":"77e9bb5c-3149-447a-ad13-c8a4ab91cad7","Type":"ContainerStarted","Data":"aee7b2a09100af4faa7c1f2c9a49149a28a37daebd2b060a4f93b5603e24e69e"} Sep 08 10:29:45.471694 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:45.471646 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-xlthg" event={"ID":"c8637904-e3b4-4b76-8908-a8fe50655647","Type":"ContainerStarted","Data":"463deb2bd62170c7bf842fdbcbd277309455e174a5bdfb5577f69dffb6f8c433"} Sep 08 10:29:45.487426 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:45.487375 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-xlthg" podStartSLOduration=2.24406841 podStartE2EDuration="4.487354999s" totalImagesPullingTime="2.243286589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:29:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:29:45.48698906 +0000 UTC m=+761.856254358" watchObservedRunningTime="2026-09-08 10:29:45.487354999 +0000 UTC m=+761.856620296" Sep 08 10:29:47.482136 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:47.482100 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-f4hmw" event={"ID":"77e9bb5c-3149-447a-ad13-c8a4ab91cad7","Type":"ContainerStarted","Data":"45a0823be753ede8f0b6a7f707e072ed58e386cd317140ed9d642e801983093e"} Sep 08 10:29:47.500018 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:47.499961 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-f4hmw" podStartSLOduration=2.279278785 podStartE2EDuration="6.499938775s" totalImagesPullingTime="4.22065999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:29:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:29:47.498510692 +0000 UTC m=+763.867775991" watchObservedRunningTime="2026-09-08 10:29:47.499938775 +0000 UTC m=+763.869204073" Sep 08 10:29:50.837867 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:50.837837 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl"] Sep 08 10:29:50.844497 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:50.844477 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:50.847104 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:50.847081 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Sep 08 10:29:50.847104 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:50.847096 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 08 10:29:50.847276 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:50.847170 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-274kd\"" Sep 08 10:29:50.847495 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:50.847480 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Sep 08 10:29:50.847593 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:50.847485 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Sep 08 10:29:50.856713 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:50.856693 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl"] Sep 08 10:29:50.971896 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:50.971871 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-registry-tls\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:50.971896 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:50.971897 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-offline-tls\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:50.972042 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:50.971979 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-online-tls\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:50.972042 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:50.972003 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-feast-data\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:50.972042 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:50.972030 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnvrl\" (UniqueName: \"kubernetes.io/projected/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-kube-api-access-hnvrl\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:51.073227 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:51.073199 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-registry-tls\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:51.073227 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:51.073229 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-offline-tls\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:51.073364 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:51.073268 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-online-tls\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:51.073364 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:29:51.073342 2561 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Sep 08 10:29:51.073364 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:29:51.073351 2561 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-offline-tls: secret "feast-credit-scoring-offline-tls" not found Sep 08 10:29:51.073457 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:51.073379 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-feast-data\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:51.073457 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:29:51.073413 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-offline-tls podName:b472dba8-7ae0-4a64-92ef-bc0d9e10b233 nodeName:}" failed. No retries permitted until 2026-09-08 10:29:51.57339162 +0000 UTC m=+767.942656915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-offline-tls") pod "feast-credit-scoring-59cf6f74d5-g74bl" (UID: "b472dba8-7ae0-4a64-92ef-bc0d9e10b233") : secret "feast-credit-scoring-offline-tls" not found Sep 08 10:29:51.073585 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:29:51.073462 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-registry-tls podName:b472dba8-7ae0-4a64-92ef-bc0d9e10b233 nodeName:}" failed. No retries permitted until 2026-09-08 10:29:51.5734402 +0000 UTC m=+767.942705478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-registry-tls") pod "feast-credit-scoring-59cf6f74d5-g74bl" (UID: "b472dba8-7ae0-4a64-92ef-bc0d9e10b233") : secret "feast-credit-scoring-registry-tls" not found Sep 08 10:29:51.073585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:51.073497 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnvrl\" (UniqueName: \"kubernetes.io/projected/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-kube-api-access-hnvrl\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:51.073768 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:51.073748 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-feast-data\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:51.075769 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:51.075747 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-online-tls\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:51.086682 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:51.086651 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnvrl\" (UniqueName: \"kubernetes.io/projected/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-kube-api-access-hnvrl\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:51.577003 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:51.576980 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-registry-tls\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:51.577142 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:51.577008 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-offline-tls\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:51.579230 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:51.579201 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-registry-tls\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:51.579360 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:51.579340 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-offline-tls\") pod \"feast-credit-scoring-59cf6f74d5-g74bl\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:51.753545 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:51.753509 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:29:52.033553 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:52.033517 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl"] Sep 08 10:29:52.034755 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:29:52.034729 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb472dba8_7ae0_4a64_92ef_bc0d9e10b233.slice/crio-98d30647a810f45d54b76ee61ebd60363f3308cf1f6d773a4fc5b70aeacbf803 WatchSource:0}: Error finding container 98d30647a810f45d54b76ee61ebd60363f3308cf1f6d773a4fc5b70aeacbf803: Status 404 returned error can't find the container with id 98d30647a810f45d54b76ee61ebd60363f3308cf1f6d773a4fc5b70aeacbf803 Sep 08 10:29:52.505161 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:52.505119 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" event={"ID":"b472dba8-7ae0-4a64-92ef-bc0d9e10b233","Type":"ContainerStarted","Data":"d9a7504608aaaa56a3748f349923783a619775adce6fb50afa42059828a01744"} Sep 08 10:29:52.505350 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:52.505185 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" event={"ID":"b472dba8-7ae0-4a64-92ef-bc0d9e10b233","Type":"ContainerStarted","Data":"98d30647a810f45d54b76ee61ebd60363f3308cf1f6d773a4fc5b70aeacbf803"} Sep 08 10:29:53.510392 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:53.510361 2561 generic.go:356] "Generic (PLEG): container finished" podID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerID="d9a7504608aaaa56a3748f349923783a619775adce6fb50afa42059828a01744" exitCode=0 Sep 08 10:29:53.510775 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:53.510446 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" event={"ID":"b472dba8-7ae0-4a64-92ef-bc0d9e10b233","Type":"ContainerDied","Data":"d9a7504608aaaa56a3748f349923783a619775adce6fb50afa42059828a01744"} Sep 08 10:29:54.516840 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:29:54.516806 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" event={"ID":"b472dba8-7ae0-4a64-92ef-bc0d9e10b233","Type":"ContainerStarted","Data":"26bc7e0018987e74bd56948b976d75298ccb852189ba2259049a7486cadd6d7e"} Sep 08 10:30:00.544022 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:00.543990 2561 generic.go:356] "Generic (PLEG): container finished" podID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerID="26bc7e0018987e74bd56948b976d75298ccb852189ba2259049a7486cadd6d7e" exitCode=0 Sep 08 10:30:00.544348 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:00.544040 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" event={"ID":"b472dba8-7ae0-4a64-92ef-bc0d9e10b233","Type":"ContainerDied","Data":"26bc7e0018987e74bd56948b976d75298ccb852189ba2259049a7486cadd6d7e"} Sep 08 10:30:01.554208 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:01.554173 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" event={"ID":"b472dba8-7ae0-4a64-92ef-bc0d9e10b233","Type":"ContainerStarted","Data":"62de76b7ff3223f5ea61d4a2dc919df9a318856a62fc5520c86bca2f8036f9e0"} Sep 08 10:30:01.554692 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:01.554236 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" event={"ID":"b472dba8-7ae0-4a64-92ef-bc0d9e10b233","Type":"ContainerStarted","Data":"cf67fd21a296537eba64389e09bead3afdc71e0059a840dd7e2036367a3ddcf0"} Sep 08 10:30:01.554692 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:01.554254 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" event={"ID":"b472dba8-7ae0-4a64-92ef-bc0d9e10b233","Type":"ContainerStarted","Data":"7bc969c0f165d7325ab0333e7b9e81daeb5b14146a7a853a391284f91b042c41"} Sep 08 10:30:01.580303 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:01.580246 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" podStartSLOduration=11.580224833 podStartE2EDuration="11.580224833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:29:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:30:01.578747547 +0000 UTC m=+777.948012847" watchObservedRunningTime="2026-09-08 10:30:01.580224833 +0000 UTC m=+777.949490134" Sep 08 10:30:01.754176 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:01.754127 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:30:01.754347 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:01.754202 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:30:01.754347 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:01.754221 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:30:03.754456 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:03.754407 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:30:03.754940 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:03.754521 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:30:03.754940 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:03.754590 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:30:03.756198 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:03.756162 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.32:6571: connect: connection refused" Sep 08 10:30:03.756312 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:03.756212 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.32:8816: connect: connection refused" Sep 08 10:30:03.756312 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:03.756250 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 08 10:30:06.754448 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:06.754400 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.32:6571: connect: connection refused" Sep 08 10:30:06.755041 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:06.754401 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.32:8816: connect: connection refused" Sep 08 10:30:06.755150 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:06.754481 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 08 10:30:09.753940 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:09.753893 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.32:6571: connect: connection refused" Sep 08 10:30:09.754815 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:09.754791 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:30:09.755701 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:09.755679 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:30:09.760679 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:09.760659 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:30:10.608245 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:10.608204 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:30:12.754092 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:12.754046 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.32:6571: connect: connection refused" Sep 08 10:30:15.754239 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:15.754209 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:30:15.755030 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:30:15.755006 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:31:18.601058 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.601016 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl"] Sep 08 10:31:18.613883 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.613848 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-f4hmw"] Sep 08 10:31:18.614060 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.614019 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-f4hmw" podUID="77e9bb5c-3149-447a-ad13-c8a4ab91cad7" containerName="postgres" containerID="cri-o://45a0823be753ede8f0b6a7f707e072ed58e386cd317140ed9d642e801983093e" gracePeriod=30 Sep 08 10:31:18.625703 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.625670 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-xlthg"] Sep 08 10:31:18.625860 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.625838 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-xlthg" podUID="c8637904-e3b4-4b76-8908-a8fe50655647" containerName="redis" containerID="cri-o://463deb2bd62170c7bf842fdbcbd277309455e174a5bdfb5577f69dffb6f8c433" gracePeriod=30 Sep 08 10:31:18.887523 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.887500 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-xlthg" Sep 08 10:31:18.890761 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.890737 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-f4hmw" Sep 08 10:31:18.909630 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.909604 2561 generic.go:356] "Generic (PLEG): container finished" podID="77e9bb5c-3149-447a-ad13-c8a4ab91cad7" containerID="45a0823be753ede8f0b6a7f707e072ed58e386cd317140ed9d642e801983093e" exitCode=0 Sep 08 10:31:18.909837 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.909641 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-f4hmw" Sep 08 10:31:18.909837 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.909696 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-f4hmw" event={"ID":"77e9bb5c-3149-447a-ad13-c8a4ab91cad7","Type":"ContainerDied","Data":"45a0823be753ede8f0b6a7f707e072ed58e386cd317140ed9d642e801983093e"} Sep 08 10:31:18.909837 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.909739 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-f4hmw" event={"ID":"77e9bb5c-3149-447a-ad13-c8a4ab91cad7","Type":"ContainerDied","Data":"aee7b2a09100af4faa7c1f2c9a49149a28a37daebd2b060a4f93b5603e24e69e"} Sep 08 10:31:18.909837 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.909762 2561 scope.go:117] "RemoveContainer" containerID="45a0823be753ede8f0b6a7f707e072ed58e386cd317140ed9d642e801983093e" Sep 08 10:31:18.911096 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.911069 2561 generic.go:356] "Generic (PLEG): container finished" podID="c8637904-e3b4-4b76-8908-a8fe50655647" containerID="463deb2bd62170c7bf842fdbcbd277309455e174a5bdfb5577f69dffb6f8c433" exitCode=0 Sep 08 10:31:18.911184 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.911105 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-xlthg" Sep 08 10:31:18.911184 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.911173 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-xlthg" event={"ID":"c8637904-e3b4-4b76-8908-a8fe50655647","Type":"ContainerDied","Data":"463deb2bd62170c7bf842fdbcbd277309455e174a5bdfb5577f69dffb6f8c433"} Sep 08 10:31:18.911285 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.911196 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-xlthg" event={"ID":"c8637904-e3b4-4b76-8908-a8fe50655647","Type":"ContainerDied","Data":"9a0dbf95a8ef249394d45d652ccbcb6ed2945b8256a9cb83479c310b79ce7c8c"} Sep 08 10:31:18.911337 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.911301 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="registry" containerID="cri-o://7bc969c0f165d7325ab0333e7b9e81daeb5b14146a7a853a391284f91b042c41" gracePeriod=30 Sep 08 10:31:18.911432 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.911415 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="offline" containerID="cri-o://62de76b7ff3223f5ea61d4a2dc919df9a318856a62fc5520c86bca2f8036f9e0" gracePeriod=30 Sep 08 10:31:18.911492 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.911459 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="online" containerID="cri-o://cf67fd21a296537eba64389e09bead3afdc71e0059a840dd7e2036367a3ddcf0" gracePeriod=30 Sep 08 10:31:18.918881 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.918852 2561 scope.go:117] "RemoveContainer" containerID="45a0823be753ede8f0b6a7f707e072ed58e386cd317140ed9d642e801983093e" Sep 08 10:31:18.919438 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:31:18.919137 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45a0823be753ede8f0b6a7f707e072ed58e386cd317140ed9d642e801983093e\": container with ID starting with 45a0823be753ede8f0b6a7f707e072ed58e386cd317140ed9d642e801983093e not found: ID does not exist" containerID="45a0823be753ede8f0b6a7f707e072ed58e386cd317140ed9d642e801983093e" Sep 08 10:31:18.919438 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.919170 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45a0823be753ede8f0b6a7f707e072ed58e386cd317140ed9d642e801983093e"} err="failed to get container status \"45a0823be753ede8f0b6a7f707e072ed58e386cd317140ed9d642e801983093e\": rpc error: code = NotFound desc = could not find container \"45a0823be753ede8f0b6a7f707e072ed58e386cd317140ed9d642e801983093e\": container with ID starting with 45a0823be753ede8f0b6a7f707e072ed58e386cd317140ed9d642e801983093e not found: ID does not exist" Sep 08 10:31:18.919438 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.919195 2561 scope.go:117] "RemoveContainer" containerID="463deb2bd62170c7bf842fdbcbd277309455e174a5bdfb5577f69dffb6f8c433" Sep 08 10:31:18.927432 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.927417 2561 scope.go:117] "RemoveContainer" containerID="463deb2bd62170c7bf842fdbcbd277309455e174a5bdfb5577f69dffb6f8c433" Sep 08 10:31:18.927727 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:31:18.927705 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"463deb2bd62170c7bf842fdbcbd277309455e174a5bdfb5577f69dffb6f8c433\": container with ID starting with 463deb2bd62170c7bf842fdbcbd277309455e174a5bdfb5577f69dffb6f8c433 not found: ID does not exist" containerID="463deb2bd62170c7bf842fdbcbd277309455e174a5bdfb5577f69dffb6f8c433" Sep 08 10:31:18.927797 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.927726 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"463deb2bd62170c7bf842fdbcbd277309455e174a5bdfb5577f69dffb6f8c433"} err="failed to get container status \"463deb2bd62170c7bf842fdbcbd277309455e174a5bdfb5577f69dffb6f8c433\": rpc error: code = NotFound desc = could not find container \"463deb2bd62170c7bf842fdbcbd277309455e174a5bdfb5577f69dffb6f8c433\": container with ID starting with 463deb2bd62170c7bf842fdbcbd277309455e174a5bdfb5577f69dffb6f8c433 not found: ID does not exist" Sep 08 10:31:18.991595 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.991571 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/77e9bb5c-3149-447a-ad13-c8a4ab91cad7-postgresdata\") pod \"77e9bb5c-3149-447a-ad13-c8a4ab91cad7\" (UID: \"77e9bb5c-3149-447a-ad13-c8a4ab91cad7\") " Sep 08 10:31:18.991749 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.991660 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wlrg\" (UniqueName: \"kubernetes.io/projected/77e9bb5c-3149-447a-ad13-c8a4ab91cad7-kube-api-access-8wlrg\") pod \"77e9bb5c-3149-447a-ad13-c8a4ab91cad7\" (UID: \"77e9bb5c-3149-447a-ad13-c8a4ab91cad7\") " Sep 08 10:31:18.991749 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.991705 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hf97\" (UniqueName: \"kubernetes.io/projected/c8637904-e3b4-4b76-8908-a8fe50655647-kube-api-access-6hf97\") pod \"c8637904-e3b4-4b76-8908-a8fe50655647\" (UID: \"c8637904-e3b4-4b76-8908-a8fe50655647\") " Sep 08 10:31:18.993548 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.993511 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77e9bb5c-3149-447a-ad13-c8a4ab91cad7-kube-api-access-8wlrg" (OuterVolumeSpecName: "kube-api-access-8wlrg") pod "77e9bb5c-3149-447a-ad13-c8a4ab91cad7" (UID: "77e9bb5c-3149-447a-ad13-c8a4ab91cad7"). InnerVolumeSpecName "kube-api-access-8wlrg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:31:18.993706 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:18.993688 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8637904-e3b4-4b76-8908-a8fe50655647-kube-api-access-6hf97" (OuterVolumeSpecName: "kube-api-access-6hf97") pod "c8637904-e3b4-4b76-8908-a8fe50655647" (UID: "c8637904-e3b4-4b76-8908-a8fe50655647"). InnerVolumeSpecName "kube-api-access-6hf97". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:31:19.020737 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:19.020705 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77e9bb5c-3149-447a-ad13-c8a4ab91cad7-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "77e9bb5c-3149-447a-ad13-c8a4ab91cad7" (UID: "77e9bb5c-3149-447a-ad13-c8a4ab91cad7"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:31:19.092445 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:19.092422 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wlrg\" (UniqueName: \"kubernetes.io/projected/77e9bb5c-3149-447a-ad13-c8a4ab91cad7-kube-api-access-8wlrg\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:31:19.092445 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:19.092444 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hf97\" (UniqueName: \"kubernetes.io/projected/c8637904-e3b4-4b76-8908-a8fe50655647-kube-api-access-6hf97\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:31:19.092588 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:19.092458 2561 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/77e9bb5c-3149-447a-ad13-c8a4ab91cad7-postgresdata\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:31:19.238434 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:19.238407 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-f4hmw"] Sep 08 10:31:19.243316 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:19.243291 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-f4hmw"] Sep 08 10:31:19.255732 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:19.255709 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-xlthg"] Sep 08 10:31:19.259988 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:19.259966 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-xlthg"] Sep 08 10:31:19.917392 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:19.917328 2561 generic.go:356] "Generic (PLEG): container finished" podID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerID="cf67fd21a296537eba64389e09bead3afdc71e0059a840dd7e2036367a3ddcf0" exitCode=0 Sep 08 10:31:19.917715 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:19.917392 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" event={"ID":"b472dba8-7ae0-4a64-92ef-bc0d9e10b233","Type":"ContainerDied","Data":"cf67fd21a296537eba64389e09bead3afdc71e0059a840dd7e2036367a3ddcf0"} Sep 08 10:31:20.144658 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:20.144630 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77e9bb5c-3149-447a-ad13-c8a4ab91cad7" path="/var/lib/kubelet/pods/77e9bb5c-3149-447a-ad13-c8a4ab91cad7/volumes" Sep 08 10:31:20.144947 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:20.144934 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8637904-e3b4-4b76-8908-a8fe50655647" path="/var/lib/kubelet/pods/c8637904-e3b4-4b76-8908-a8fe50655647/volumes" Sep 08 10:31:20.605656 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:20.605622 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 08 10:31:30.605391 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:30.605345 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 08 10:31:40.605231 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:40.605178 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Sep 08 10:31:40.605656 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:40.605310 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:31:49.046601 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.046571 2561 generic.go:356] "Generic (PLEG): container finished" podID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerID="62de76b7ff3223f5ea61d4a2dc919df9a318856a62fc5520c86bca2f8036f9e0" exitCode=137 Sep 08 10:31:49.046601 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.046598 2561 generic.go:356] "Generic (PLEG): container finished" podID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerID="7bc969c0f165d7325ab0333e7b9e81daeb5b14146a7a853a391284f91b042c41" exitCode=137 Sep 08 10:31:49.046916 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.046646 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" event={"ID":"b472dba8-7ae0-4a64-92ef-bc0d9e10b233","Type":"ContainerDied","Data":"62de76b7ff3223f5ea61d4a2dc919df9a318856a62fc5520c86bca2f8036f9e0"} Sep 08 10:31:49.046916 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.046700 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" event={"ID":"b472dba8-7ae0-4a64-92ef-bc0d9e10b233","Type":"ContainerDied","Data":"7bc969c0f165d7325ab0333e7b9e81daeb5b14146a7a853a391284f91b042c41"} Sep 08 10:31:49.556406 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.556382 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:31:49.724035 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.723969 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-registry-tls\") pod \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " Sep 08 10:31:49.724035 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.724016 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-offline-tls\") pod \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " Sep 08 10:31:49.724318 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.724043 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnvrl\" (UniqueName: \"kubernetes.io/projected/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-kube-api-access-hnvrl\") pod \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " Sep 08 10:31:49.724318 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.724102 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-feast-data\") pod \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " Sep 08 10:31:49.724318 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.724128 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-online-tls\") pod \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\" (UID: \"b472dba8-7ae0-4a64-92ef-bc0d9e10b233\") " Sep 08 10:31:49.726241 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.726208 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-online-tls" (OuterVolumeSpecName: "online-tls") pod "b472dba8-7ae0-4a64-92ef-bc0d9e10b233" (UID: "b472dba8-7ae0-4a64-92ef-bc0d9e10b233"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:31:49.726395 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.726336 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b472dba8-7ae0-4a64-92ef-bc0d9e10b233" (UID: "b472dba8-7ae0-4a64-92ef-bc0d9e10b233"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:31:49.726741 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.726714 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "b472dba8-7ae0-4a64-92ef-bc0d9e10b233" (UID: "b472dba8-7ae0-4a64-92ef-bc0d9e10b233"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:31:49.726805 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.726724 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-kube-api-access-hnvrl" (OuterVolumeSpecName: "kube-api-access-hnvrl") pod "b472dba8-7ae0-4a64-92ef-bc0d9e10b233" (UID: "b472dba8-7ae0-4a64-92ef-bc0d9e10b233"). InnerVolumeSpecName "kube-api-access-hnvrl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:31:49.735598 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.735573 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-feast-data" (OuterVolumeSpecName: "feast-data") pod "b472dba8-7ae0-4a64-92ef-bc0d9e10b233" (UID: "b472dba8-7ae0-4a64-92ef-bc0d9e10b233"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:31:49.825228 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.825201 2561 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-feast-data\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:31:49.825320 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.825241 2561 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-online-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:31:49.825320 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.825251 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-registry-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:31:49.825320 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.825260 2561 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-offline-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:31:49.825320 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:49.825269 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnvrl\" (UniqueName: \"kubernetes.io/projected/b472dba8-7ae0-4a64-92ef-bc0d9e10b233-kube-api-access-hnvrl\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:31:50.053104 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:50.053076 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" event={"ID":"b472dba8-7ae0-4a64-92ef-bc0d9e10b233","Type":"ContainerDied","Data":"98d30647a810f45d54b76ee61ebd60363f3308cf1f6d773a4fc5b70aeacbf803"} Sep 08 10:31:50.053437 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:50.053105 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl" Sep 08 10:31:50.053437 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:50.053140 2561 scope.go:117] "RemoveContainer" containerID="62de76b7ff3223f5ea61d4a2dc919df9a318856a62fc5520c86bca2f8036f9e0" Sep 08 10:31:50.061894 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:50.061876 2561 scope.go:117] "RemoveContainer" containerID="cf67fd21a296537eba64389e09bead3afdc71e0059a840dd7e2036367a3ddcf0" Sep 08 10:31:50.074288 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:50.074268 2561 scope.go:117] "RemoveContainer" containerID="7bc969c0f165d7325ab0333e7b9e81daeb5b14146a7a853a391284f91b042c41" Sep 08 10:31:50.082550 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:50.082516 2561 scope.go:117] "RemoveContainer" containerID="26bc7e0018987e74bd56948b976d75298ccb852189ba2259049a7486cadd6d7e" Sep 08 10:31:50.091204 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:50.091184 2561 scope.go:117] "RemoveContainer" containerID="d9a7504608aaaa56a3748f349923783a619775adce6fb50afa42059828a01744" Sep 08 10:31:50.094147 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:50.094093 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl"] Sep 08 10:31:50.096027 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:50.096006 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-59cf6f74d5-g74bl"] Sep 08 10:31:50.144789 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:31:50.144768 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" path="/var/lib/kubelet/pods/b472dba8-7ae0-4a64-92ef-bc0d9e10b233/volumes" Sep 08 10:33:56.194986 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.194890 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-xcx5s"] Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195410 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8637904-e3b4-4b76-8908-a8fe50655647" containerName="redis" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195434 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8637904-e3b4-4b76-8908-a8fe50655647" containerName="redis" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195447 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="online" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195454 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="online" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195469 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="feast-apply" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195475 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="feast-apply" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195489 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77e9bb5c-3149-447a-ad13-c8a4ab91cad7" containerName="postgres" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195496 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="77e9bb5c-3149-447a-ad13-c8a4ab91cad7" containerName="postgres" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195505 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="feast-init" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195511 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="feast-init" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195517 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="registry" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195546 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="registry" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195553 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="offline" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195558 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="offline" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195625 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="offline" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195634 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="registry" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195646 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="77e9bb5c-3149-447a-ad13-c8a4ab91cad7" containerName="postgres" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195653 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8637904-e3b4-4b76-8908-a8fe50655647" containerName="redis" Sep 08 10:33:56.197585 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.195658 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b472dba8-7ae0-4a64-92ef-bc0d9e10b233" containerName="online" Sep 08 10:33:56.198734 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.198711 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-xcx5s" Sep 08 10:33:56.201097 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.201074 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 08 10:33:56.202159 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.202135 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 08 10:33:56.202281 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.202153 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-tgpsh\"" Sep 08 10:33:56.219110 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.219087 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6rdv\" (UniqueName: \"kubernetes.io/projected/5e87bd1c-379f-48a5-afa2-e412963bb7b6-kube-api-access-l6rdv\") pod \"redis-69bbd798b6-xcx5s\" (UID: \"5e87bd1c-379f-48a5-afa2-e412963bb7b6\") " pod="test-ns-feast-rest/redis-69bbd798b6-xcx5s" Sep 08 10:33:56.220820 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.220795 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-xcx5s"] Sep 08 10:33:56.319671 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.319644 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6rdv\" (UniqueName: \"kubernetes.io/projected/5e87bd1c-379f-48a5-afa2-e412963bb7b6-kube-api-access-l6rdv\") pod \"redis-69bbd798b6-xcx5s\" (UID: \"5e87bd1c-379f-48a5-afa2-e412963bb7b6\") " pod="test-ns-feast-rest/redis-69bbd798b6-xcx5s" Sep 08 10:33:56.329099 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.329079 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6rdv\" (UniqueName: \"kubernetes.io/projected/5e87bd1c-379f-48a5-afa2-e412963bb7b6-kube-api-access-l6rdv\") pod \"redis-69bbd798b6-xcx5s\" (UID: \"5e87bd1c-379f-48a5-afa2-e412963bb7b6\") " pod="test-ns-feast-rest/redis-69bbd798b6-xcx5s" Sep 08 10:33:56.516149 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.516124 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-xcx5s" Sep 08 10:33:56.784226 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:56.784160 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-xcx5s"] Sep 08 10:33:56.787391 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:33:56.787363 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e87bd1c_379f_48a5_afa2_e412963bb7b6.slice/crio-9f84b61fe48967771f9c8327e8b9089eb1a09761336ac8a2160a2c05730836b6 WatchSource:0}: Error finding container 9f84b61fe48967771f9c8327e8b9089eb1a09761336ac8a2160a2c05730836b6: Status 404 returned error can't find the container with id 9f84b61fe48967771f9c8327e8b9089eb1a09761336ac8a2160a2c05730836b6 Sep 08 10:33:57.576414 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:57.576374 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-xcx5s" event={"ID":"5e87bd1c-379f-48a5-afa2-e412963bb7b6","Type":"ContainerStarted","Data":"7f5f50e16fb84b40ab4002d3ed06c35649e2705efe16a4a3f5073a7c78a9540d"} Sep 08 10:33:57.576414 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:57.576418 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-xcx5s" event={"ID":"5e87bd1c-379f-48a5-afa2-e412963bb7b6","Type":"ContainerStarted","Data":"9f84b61fe48967771f9c8327e8b9089eb1a09761336ac8a2160a2c05730836b6"} Sep 08 10:33:57.596156 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:33:57.596110 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-xcx5s" podStartSLOduration=1.596095362 podStartE2EDuration="1.596095362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:33:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:33:57.594867962 +0000 UTC m=+1013.964133259" watchObservedRunningTime="2026-09-08 10:33:57.596095362 +0000 UTC m=+1013.965360711" Sep 08 10:34:01.650542 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:01.650492 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-n5stc"] Sep 08 10:34:01.653852 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:01.653830 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-n5stc" Sep 08 10:34:01.656437 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:01.656415 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 08 10:34:01.664754 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:01.664732 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-n5stc"] Sep 08 10:34:01.763365 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:01.763340 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h94h9\" (UniqueName: \"kubernetes.io/projected/fad98518-bf57-47ab-b0cc-c7cfb2550251-kube-api-access-h94h9\") pod \"postgres-864846d4f5-n5stc\" (UID: \"fad98518-bf57-47ab-b0cc-c7cfb2550251\") " pod="test-ns-feast-rest/postgres-864846d4f5-n5stc" Sep 08 10:34:01.763465 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:01.763375 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/fad98518-bf57-47ab-b0cc-c7cfb2550251-postgresdata\") pod \"postgres-864846d4f5-n5stc\" (UID: \"fad98518-bf57-47ab-b0cc-c7cfb2550251\") " pod="test-ns-feast-rest/postgres-864846d4f5-n5stc" Sep 08 10:34:01.864515 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:01.864489 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h94h9\" (UniqueName: \"kubernetes.io/projected/fad98518-bf57-47ab-b0cc-c7cfb2550251-kube-api-access-h94h9\") pod \"postgres-864846d4f5-n5stc\" (UID: \"fad98518-bf57-47ab-b0cc-c7cfb2550251\") " pod="test-ns-feast-rest/postgres-864846d4f5-n5stc" Sep 08 10:34:01.864648 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:01.864541 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/fad98518-bf57-47ab-b0cc-c7cfb2550251-postgresdata\") pod \"postgres-864846d4f5-n5stc\" (UID: \"fad98518-bf57-47ab-b0cc-c7cfb2550251\") " pod="test-ns-feast-rest/postgres-864846d4f5-n5stc" Sep 08 10:34:01.864862 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:01.864844 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/fad98518-bf57-47ab-b0cc-c7cfb2550251-postgresdata\") pod \"postgres-864846d4f5-n5stc\" (UID: \"fad98518-bf57-47ab-b0cc-c7cfb2550251\") " pod="test-ns-feast-rest/postgres-864846d4f5-n5stc" Sep 08 10:34:01.874103 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:01.874083 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h94h9\" (UniqueName: \"kubernetes.io/projected/fad98518-bf57-47ab-b0cc-c7cfb2550251-kube-api-access-h94h9\") pod \"postgres-864846d4f5-n5stc\" (UID: \"fad98518-bf57-47ab-b0cc-c7cfb2550251\") " pod="test-ns-feast-rest/postgres-864846d4f5-n5stc" Sep 08 10:34:01.963173 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:01.963113 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-n5stc" Sep 08 10:34:02.235430 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:02.235354 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-n5stc"] Sep 08 10:34:02.239044 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:34:02.239017 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfad98518_bf57_47ab_b0cc_c7cfb2550251.slice/crio-536d68380cc45453e39c99a717d9ddd1de1ac604be32ec96ebbbc6e2d62d3d9a WatchSource:0}: Error finding container 536d68380cc45453e39c99a717d9ddd1de1ac604be32ec96ebbbc6e2d62d3d9a: Status 404 returned error can't find the container with id 536d68380cc45453e39c99a717d9ddd1de1ac604be32ec96ebbbc6e2d62d3d9a Sep 08 10:34:02.605955 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:02.605908 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-n5stc" event={"ID":"fad98518-bf57-47ab-b0cc-c7cfb2550251","Type":"ContainerStarted","Data":"a6852a407519b1468b708ff334798a7698702fd5014c5f9029fe15e004091117"} Sep 08 10:34:02.606107 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:02.605965 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-n5stc" event={"ID":"fad98518-bf57-47ab-b0cc-c7cfb2550251","Type":"ContainerStarted","Data":"536d68380cc45453e39c99a717d9ddd1de1ac604be32ec96ebbbc6e2d62d3d9a"} Sep 08 10:34:02.624556 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:02.624494 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-n5stc" podStartSLOduration=1.624479235 podStartE2EDuration="1.624479235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:34:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:34:02.622751881 +0000 UTC m=+1018.992017180" watchObservedRunningTime="2026-09-08 10:34:02.624479235 +0000 UTC m=+1018.993744531" Sep 08 10:34:07.254761 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.254727 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246"] Sep 08 10:34:07.258246 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.258223 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:07.260266 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.260242 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 08 10:34:07.260616 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.260589 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-gjg52\"" Sep 08 10:34:07.260723 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.260624 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Sep 08 10:34:07.260723 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.260627 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Sep 08 10:34:07.270723 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.270704 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246"] Sep 08 10:34:07.308814 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.308791 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-online-tls\") pod \"feast-credit-scoring-fc449447c-n6246\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:07.308891 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.308818 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-registry-tls\") pod \"feast-credit-scoring-fc449447c-n6246\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:07.308891 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.308851 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-feast-data\") pod \"feast-credit-scoring-fc449447c-n6246\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:07.308973 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.308924 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tljn9\" (UniqueName: \"kubernetes.io/projected/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-kube-api-access-tljn9\") pod \"feast-credit-scoring-fc449447c-n6246\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:07.409847 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.409829 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-online-tls\") pod \"feast-credit-scoring-fc449447c-n6246\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:07.409939 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.409854 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-registry-tls\") pod \"feast-credit-scoring-fc449447c-n6246\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:07.409939 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.409892 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-feast-data\") pod \"feast-credit-scoring-fc449447c-n6246\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:07.409939 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.409924 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tljn9\" (UniqueName: \"kubernetes.io/projected/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-kube-api-access-tljn9\") pod \"feast-credit-scoring-fc449447c-n6246\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:07.410108 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:34:07.410076 2561 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Sep 08 10:34:07.410168 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:34:07.410125 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-registry-tls podName:83f0fd7a-68bf-46b0-89d3-38f18c26cdde nodeName:}" failed. No retries permitted until 2026-09-08 10:34:07.910110501 +0000 UTC m=+1024.279375790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-registry-tls") pod "feast-credit-scoring-fc449447c-n6246" (UID: "83f0fd7a-68bf-46b0-89d3-38f18c26cdde") : secret "feast-credit-scoring-registry-tls" not found Sep 08 10:34:07.410296 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.410276 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-feast-data\") pod \"feast-credit-scoring-fc449447c-n6246\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:07.412139 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.412122 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-online-tls\") pod \"feast-credit-scoring-fc449447c-n6246\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:07.419044 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.419025 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tljn9\" (UniqueName: \"kubernetes.io/projected/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-kube-api-access-tljn9\") pod \"feast-credit-scoring-fc449447c-n6246\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:07.913751 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:07.913721 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-registry-tls\") pod \"feast-credit-scoring-fc449447c-n6246\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:07.913903 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:34:07.913861 2561 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Sep 08 10:34:07.913956 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:34:07.913930 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-registry-tls podName:83f0fd7a-68bf-46b0-89d3-38f18c26cdde nodeName:}" failed. No retries permitted until 2026-09-08 10:34:08.913911852 +0000 UTC m=+1025.283177133 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-registry-tls") pod "feast-credit-scoring-fc449447c-n6246" (UID: "83f0fd7a-68bf-46b0-89d3-38f18c26cdde") : secret "feast-credit-scoring-registry-tls" not found Sep 08 10:34:08.921510 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:08.921461 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-registry-tls\") pod \"feast-credit-scoring-fc449447c-n6246\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:08.924301 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:08.924272 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-registry-tls\") pod \"feast-credit-scoring-fc449447c-n6246\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:09.067589 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:09.067555 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:09.361481 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:09.361457 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246"] Sep 08 10:34:09.364352 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:34:09.364321 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83f0fd7a_68bf_46b0_89d3_38f18c26cdde.slice/crio-3cf0f65d75ba19d38c1b97ecb2f12e18e10fc8859a2725c6a5971920616251fa WatchSource:0}: Error finding container 3cf0f65d75ba19d38c1b97ecb2f12e18e10fc8859a2725c6a5971920616251fa: Status 404 returned error can't find the container with id 3cf0f65d75ba19d38c1b97ecb2f12e18e10fc8859a2725c6a5971920616251fa Sep 08 10:34:09.635144 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:09.635054 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" event={"ID":"83f0fd7a-68bf-46b0-89d3-38f18c26cdde","Type":"ContainerStarted","Data":"9c2c56f208650852c707c7969d109e48e9f8396a6ddca9415d2b6ecf37df2da7"} Sep 08 10:34:09.635144 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:09.635115 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" event={"ID":"83f0fd7a-68bf-46b0-89d3-38f18c26cdde","Type":"ContainerStarted","Data":"3cf0f65d75ba19d38c1b97ecb2f12e18e10fc8859a2725c6a5971920616251fa"} Sep 08 10:34:10.640662 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:10.640632 2561 generic.go:356] "Generic (PLEG): container finished" podID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerID="9c2c56f208650852c707c7969d109e48e9f8396a6ddca9415d2b6ecf37df2da7" exitCode=0 Sep 08 10:34:10.640999 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:10.640720 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" event={"ID":"83f0fd7a-68bf-46b0-89d3-38f18c26cdde","Type":"ContainerDied","Data":"9c2c56f208650852c707c7969d109e48e9f8396a6ddca9415d2b6ecf37df2da7"} Sep 08 10:34:11.647081 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:11.647050 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" event={"ID":"83f0fd7a-68bf-46b0-89d3-38f18c26cdde","Type":"ContainerStarted","Data":"7085ab85cfd433585fb1086cdb6178ee3228612897ed49bedba2ef9786800268"} Sep 08 10:34:18.681170 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:18.681131 2561 generic.go:356] "Generic (PLEG): container finished" podID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerID="7085ab85cfd433585fb1086cdb6178ee3228612897ed49bedba2ef9786800268" exitCode=0 Sep 08 10:34:18.681587 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:18.681184 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" event={"ID":"83f0fd7a-68bf-46b0-89d3-38f18c26cdde","Type":"ContainerDied","Data":"7085ab85cfd433585fb1086cdb6178ee3228612897ed49bedba2ef9786800268"} Sep 08 10:34:19.689402 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:19.689369 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" event={"ID":"83f0fd7a-68bf-46b0-89d3-38f18c26cdde","Type":"ContainerStarted","Data":"f6d48d31bc5cb104d10b040e4914864b1e4cd7b53364af6aaedee025ac0acbbb"} Sep 08 10:34:19.689787 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:19.689422 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" event={"ID":"83f0fd7a-68bf-46b0-89d3-38f18c26cdde","Type":"ContainerStarted","Data":"34b0df01e123359bce1c9cef1c7b21bfc65d74d9f63a890f46702cfd48997367"} Sep 08 10:34:19.712867 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:19.712813 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" podStartSLOduration=12.712794903 podStartE2EDuration="12.712794903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:34:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:34:19.711393127 +0000 UTC m=+1036.080658425" watchObservedRunningTime="2026-09-08 10:34:19.712794903 +0000 UTC m=+1036.082060206" Sep 08 10:34:21.067762 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:21.067723 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:21.068127 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:21.067802 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:21.069377 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:21.069338 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused" Sep 08 10:34:21.069494 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:21.069383 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Sep 08 10:34:24.068003 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:24.067949 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused" Sep 08 10:34:24.068003 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:24.067977 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Sep 08 10:34:27.068593 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:27.068553 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused" Sep 08 10:34:27.074403 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:27.074366 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:27.074790 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:27.074768 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:27.077703 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:27.077679 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:29.068184 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:29.068146 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:30.068047 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:30.068007 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused" Sep 08 10:34:33.068493 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:33.068458 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:33.069334 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:33.069309 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:34:38.314053 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.314004 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94"] Sep 08 10:34:38.319297 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.319276 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:38.321814 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.321789 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Sep 08 10:34:38.321918 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.321802 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Sep 08 10:34:38.321981 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.321931 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-sjfnn\"" Sep 08 10:34:38.321981 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.321956 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Sep 08 10:34:38.332024 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.332005 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94"] Sep 08 10:34:38.457772 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.457746 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3c9527d9-0a81-4e05-bb4d-3415952554f9-feast-data\") pod \"feast-driver-ranking-7986c57686-64h94\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:38.457876 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.457776 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3c9527d9-0a81-4e05-bb4d-3415952554f9-online-tls\") pod \"feast-driver-ranking-7986c57686-64h94\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:38.457876 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.457822 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpj69\" (UniqueName: \"kubernetes.io/projected/3c9527d9-0a81-4e05-bb4d-3415952554f9-kube-api-access-cpj69\") pod \"feast-driver-ranking-7986c57686-64h94\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:38.457876 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.457854 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3c9527d9-0a81-4e05-bb4d-3415952554f9-registry-tls\") pod \"feast-driver-ranking-7986c57686-64h94\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:38.558386 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.558362 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3c9527d9-0a81-4e05-bb4d-3415952554f9-feast-data\") pod \"feast-driver-ranking-7986c57686-64h94\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:38.558479 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.558391 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3c9527d9-0a81-4e05-bb4d-3415952554f9-online-tls\") pod \"feast-driver-ranking-7986c57686-64h94\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:38.558479 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.558436 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpj69\" (UniqueName: \"kubernetes.io/projected/3c9527d9-0a81-4e05-bb4d-3415952554f9-kube-api-access-cpj69\") pod \"feast-driver-ranking-7986c57686-64h94\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:38.558479 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.558455 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3c9527d9-0a81-4e05-bb4d-3415952554f9-registry-tls\") pod \"feast-driver-ranking-7986c57686-64h94\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:38.558673 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:34:38.558599 2561 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-online-tls: secret "feast-driver-ranking-online-tls" not found Sep 08 10:34:38.558716 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:34:38.558675 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c9527d9-0a81-4e05-bb4d-3415952554f9-online-tls podName:3c9527d9-0a81-4e05-bb4d-3415952554f9 nodeName:}" failed. No retries permitted until 2026-09-08 10:34:39.058658767 +0000 UTC m=+1055.427924042 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/3c9527d9-0a81-4e05-bb4d-3415952554f9-online-tls") pod "feast-driver-ranking-7986c57686-64h94" (UID: "3c9527d9-0a81-4e05-bb4d-3415952554f9") : secret "feast-driver-ranking-online-tls" not found Sep 08 10:34:38.558816 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.558794 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3c9527d9-0a81-4e05-bb4d-3415952554f9-feast-data\") pod \"feast-driver-ranking-7986c57686-64h94\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:38.560730 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.560706 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3c9527d9-0a81-4e05-bb4d-3415952554f9-registry-tls\") pod \"feast-driver-ranking-7986c57686-64h94\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:38.571728 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:38.571673 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpj69\" (UniqueName: \"kubernetes.io/projected/3c9527d9-0a81-4e05-bb4d-3415952554f9-kube-api-access-cpj69\") pod \"feast-driver-ranking-7986c57686-64h94\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:39.062011 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:39.061977 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3c9527d9-0a81-4e05-bb4d-3415952554f9-online-tls\") pod \"feast-driver-ranking-7986c57686-64h94\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:39.064265 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:39.064242 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3c9527d9-0a81-4e05-bb4d-3415952554f9-online-tls\") pod \"feast-driver-ranking-7986c57686-64h94\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:39.228869 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:39.228841 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:39.502832 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:39.502805 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94"] Sep 08 10:34:39.504686 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:34:39.504661 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c9527d9_0a81_4e05_bb4d_3415952554f9.slice/crio-9048b3bb229535f97abe8bec9e0fb3efdd8af5edd546afb0a18909dd84d88a92 WatchSource:0}: Error finding container 9048b3bb229535f97abe8bec9e0fb3efdd8af5edd546afb0a18909dd84d88a92: Status 404 returned error can't find the container with id 9048b3bb229535f97abe8bec9e0fb3efdd8af5edd546afb0a18909dd84d88a92 Sep 08 10:34:39.782966 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:39.782873 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" event={"ID":"3c9527d9-0a81-4e05-bb4d-3415952554f9","Type":"ContainerStarted","Data":"9713f076ffd4c7652515fffced4baa16342febff3f877ee6cdd42c3c0e8c4204"} Sep 08 10:34:39.782966 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:39.782939 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" event={"ID":"3c9527d9-0a81-4e05-bb4d-3415952554f9","Type":"ContainerStarted","Data":"9048b3bb229535f97abe8bec9e0fb3efdd8af5edd546afb0a18909dd84d88a92"} Sep 08 10:34:44.806325 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:44.806285 2561 generic.go:356] "Generic (PLEG): container finished" podID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerID="9713f076ffd4c7652515fffced4baa16342febff3f877ee6cdd42c3c0e8c4204" exitCode=0 Sep 08 10:34:44.806756 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:44.806365 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" event={"ID":"3c9527d9-0a81-4e05-bb4d-3415952554f9","Type":"ContainerDied","Data":"9713f076ffd4c7652515fffced4baa16342febff3f877ee6cdd42c3c0e8c4204"} Sep 08 10:34:45.812459 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:45.812428 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" event={"ID":"3c9527d9-0a81-4e05-bb4d-3415952554f9","Type":"ContainerStarted","Data":"7399688fa40a30a7c03669ccee97a98b8a2ecb9796654976fa7f68d6b22e7247"} Sep 08 10:34:51.844403 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:51.844361 2561 generic.go:356] "Generic (PLEG): container finished" podID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerID="7399688fa40a30a7c03669ccee97a98b8a2ecb9796654976fa7f68d6b22e7247" exitCode=0 Sep 08 10:34:51.844884 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:51.844430 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" event={"ID":"3c9527d9-0a81-4e05-bb4d-3415952554f9","Type":"ContainerDied","Data":"7399688fa40a30a7c03669ccee97a98b8a2ecb9796654976fa7f68d6b22e7247"} Sep 08 10:34:52.852271 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:52.852232 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" event={"ID":"3c9527d9-0a81-4e05-bb4d-3415952554f9","Type":"ContainerStarted","Data":"ccd183419451b6ed4d6ddc356a5eacca076fc20ed1d0d8bfd44e6568a28e1533"} Sep 08 10:34:52.852684 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:52.852308 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" event={"ID":"3c9527d9-0a81-4e05-bb4d-3415952554f9","Type":"ContainerStarted","Data":"dc9d4b66e96950d5ac34ea767a62bd0e35d558eef0ec058a3323cb4483c1e3c9"} Sep 08 10:34:52.876513 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:52.876458 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" podStartSLOduration=14.876442334 podStartE2EDuration="14.876442334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:34:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:34:52.874941982 +0000 UTC m=+1069.244207280" watchObservedRunningTime="2026-09-08 10:34:52.876442334 +0000 UTC m=+1069.245707630" Sep 08 10:34:54.229625 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:54.229574 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:54.230023 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:54.229665 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:54.231396 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:54.231365 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.36:6571: connect: connection refused" Sep 08 10:34:54.231538 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:54.231405 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="online" probeResult="failure" output="Get \"https://10.132.0.36:6567/health\": dial tcp 10.132.0.36:6567: connect: connection refused" Sep 08 10:34:57.230031 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:57.229928 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.36:6571: connect: connection refused" Sep 08 10:34:57.230031 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:57.229971 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="online" probeResult="failure" output="Get \"https://10.132.0.36:6567/health\": dial tcp 10.132.0.36:6567: connect: connection refused" Sep 08 10:34:59.229355 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:59.229316 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:34:59.229773 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:34:59.229433 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:35:00.229622 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:35:00.229567 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.36:6571: connect: connection refused" Sep 08 10:35:00.236173 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:35:00.236135 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:35:00.898266 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:35:00.898230 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:35:03.229798 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:35:03.229743 2561 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.36:6571: connect: connection refused" Sep 08 10:35:06.230353 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:35:06.230319 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:35:06.231251 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:35:06.231225 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:36:28.900833 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:28.900756 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq"] Sep 08 10:36:28.901285 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:28.900977 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq" podUID="81540954-a1ae-44d3-88b5-43f2d0c9d186" containerName="manager" containerID="cri-o://411173ee6da70811fc18439f9ed09d95a2bc003e6cf553a0b12c06e0b54639f2" gracePeriod=10 Sep 08 10:36:29.186576 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.186542 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq" Sep 08 10:36:29.198271 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.197977 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246"] Sep 08 10:36:29.198457 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.198280 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="registry" containerID="cri-o://34b0df01e123359bce1c9cef1c7b21bfc65d74d9f63a890f46702cfd48997367" gracePeriod=30 Sep 08 10:36:29.198457 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.198346 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="online" containerID="cri-o://f6d48d31bc5cb104d10b040e4914864b1e4cd7b53364af6aaedee025ac0acbbb" gracePeriod=30 Sep 08 10:36:29.200710 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.200684 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94"] Sep 08 10:36:29.201197 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.201121 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="registry" containerID="cri-o://dc9d4b66e96950d5ac34ea767a62bd0e35d558eef0ec058a3323cb4483c1e3c9" gracePeriod=30 Sep 08 10:36:29.201417 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.201178 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="online" containerID="cri-o://ccd183419451b6ed4d6ddc356a5eacca076fc20ed1d0d8bfd44e6568a28e1533" gracePeriod=30 Sep 08 10:36:29.287079 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.287040 2561 generic.go:356] "Generic (PLEG): container finished" podID="81540954-a1ae-44d3-88b5-43f2d0c9d186" containerID="411173ee6da70811fc18439f9ed09d95a2bc003e6cf553a0b12c06e0b54639f2" exitCode=0 Sep 08 10:36:29.287411 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.287115 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq" Sep 08 10:36:29.287411 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.287135 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq" event={"ID":"81540954-a1ae-44d3-88b5-43f2d0c9d186","Type":"ContainerDied","Data":"411173ee6da70811fc18439f9ed09d95a2bc003e6cf553a0b12c06e0b54639f2"} Sep 08 10:36:29.287411 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.287211 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq" event={"ID":"81540954-a1ae-44d3-88b5-43f2d0c9d186","Type":"ContainerDied","Data":"9ccd118cdd482c6aa63fd70f76a12629b6b528c09ff6e0e74c919d39861ef06c"} Sep 08 10:36:29.287411 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.287244 2561 scope.go:117] "RemoveContainer" containerID="411173ee6da70811fc18439f9ed09d95a2bc003e6cf553a0b12c06e0b54639f2" Sep 08 10:36:29.296738 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.296715 2561 scope.go:117] "RemoveContainer" containerID="411173ee6da70811fc18439f9ed09d95a2bc003e6cf553a0b12c06e0b54639f2" Sep 08 10:36:29.297015 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:36:29.296995 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"411173ee6da70811fc18439f9ed09d95a2bc003e6cf553a0b12c06e0b54639f2\": container with ID starting with 411173ee6da70811fc18439f9ed09d95a2bc003e6cf553a0b12c06e0b54639f2 not found: ID does not exist" containerID="411173ee6da70811fc18439f9ed09d95a2bc003e6cf553a0b12c06e0b54639f2" Sep 08 10:36:29.297063 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.297025 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"411173ee6da70811fc18439f9ed09d95a2bc003e6cf553a0b12c06e0b54639f2"} err="failed to get container status \"411173ee6da70811fc18439f9ed09d95a2bc003e6cf553a0b12c06e0b54639f2\": rpc error: code = NotFound desc = could not find container \"411173ee6da70811fc18439f9ed09d95a2bc003e6cf553a0b12c06e0b54639f2\": container with ID starting with 411173ee6da70811fc18439f9ed09d95a2bc003e6cf553a0b12c06e0b54639f2 not found: ID does not exist" Sep 08 10:36:29.306296 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.306276 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v665z\" (UniqueName: \"kubernetes.io/projected/81540954-a1ae-44d3-88b5-43f2d0c9d186-kube-api-access-v665z\") pod \"81540954-a1ae-44d3-88b5-43f2d0c9d186\" (UID: \"81540954-a1ae-44d3-88b5-43f2d0c9d186\") " Sep 08 10:36:29.308391 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.308367 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81540954-a1ae-44d3-88b5-43f2d0c9d186-kube-api-access-v665z" (OuterVolumeSpecName: "kube-api-access-v665z") pod "81540954-a1ae-44d3-88b5-43f2d0c9d186" (UID: "81540954-a1ae-44d3-88b5-43f2d0c9d186"). InnerVolumeSpecName "kube-api-access-v665z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:36:29.407748 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.407710 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v665z\" (UniqueName: \"kubernetes.io/projected/81540954-a1ae-44d3-88b5-43f2d0c9d186-kube-api-access-v665z\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:36:29.613210 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.613165 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq"] Sep 08 10:36:29.616407 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:29.616374 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bc89c8fcd-d7knq"] Sep 08 10:36:30.146225 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:30.146179 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81540954-a1ae-44d3-88b5-43f2d0c9d186" path="/var/lib/kubelet/pods/81540954-a1ae-44d3-88b5-43f2d0c9d186/volumes" Sep 08 10:36:30.294007 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:30.293970 2561 generic.go:356] "Generic (PLEG): container finished" podID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerID="ccd183419451b6ed4d6ddc356a5eacca076fc20ed1d0d8bfd44e6568a28e1533" exitCode=0 Sep 08 10:36:30.294194 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:30.294045 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" event={"ID":"3c9527d9-0a81-4e05-bb4d-3415952554f9","Type":"ContainerDied","Data":"ccd183419451b6ed4d6ddc356a5eacca076fc20ed1d0d8bfd44e6568a28e1533"} Sep 08 10:36:30.297237 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:30.297207 2561 generic.go:356] "Generic (PLEG): container finished" podID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerID="f6d48d31bc5cb104d10b040e4914864b1e4cd7b53364af6aaedee025ac0acbbb" exitCode=0 Sep 08 10:36:30.297388 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:30.297245 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" event={"ID":"83f0fd7a-68bf-46b0-89d3-38f18c26cdde","Type":"ContainerDied","Data":"f6d48d31bc5cb104d10b040e4914864b1e4cd7b53364af6aaedee025ac0acbbb"} Sep 08 10:36:30.895126 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:30.895081 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="online" probeResult="failure" output="Get \"https://10.132.0.36:6567/health\": dial tcp 10.132.0.36:6567: connect: connection refused" Sep 08 10:36:30.988107 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:30.988062 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-n5stc"] Sep 08 10:36:30.988613 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:30.988576 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-n5stc" podUID="fad98518-bf57-47ab-b0cc-c7cfb2550251" containerName="postgres" containerID="cri-o://a6852a407519b1468b708ff334798a7698702fd5014c5f9029fe15e004091117" gracePeriod=30 Sep 08 10:36:30.994064 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:30.994035 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-xcx5s"] Sep 08 10:36:30.994241 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:30.994216 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-xcx5s" podUID="5e87bd1c-379f-48a5-afa2-e412963bb7b6" containerName="redis" containerID="cri-o://7f5f50e16fb84b40ab4002d3ed06c35649e2705efe16a4a3f5073a7c78a9540d" gracePeriod=30 Sep 08 10:36:31.264561 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.264518 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-n5stc" Sep 08 10:36:31.267454 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.267428 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-xcx5s" Sep 08 10:36:31.307958 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.307899 2561 generic.go:356] "Generic (PLEG): container finished" podID="fad98518-bf57-47ab-b0cc-c7cfb2550251" containerID="a6852a407519b1468b708ff334798a7698702fd5014c5f9029fe15e004091117" exitCode=0 Sep 08 10:36:31.308142 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.308029 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-n5stc" event={"ID":"fad98518-bf57-47ab-b0cc-c7cfb2550251","Type":"ContainerDied","Data":"a6852a407519b1468b708ff334798a7698702fd5014c5f9029fe15e004091117"} Sep 08 10:36:31.308142 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.308071 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-n5stc" event={"ID":"fad98518-bf57-47ab-b0cc-c7cfb2550251","Type":"ContainerDied","Data":"536d68380cc45453e39c99a717d9ddd1de1ac604be32ec96ebbbc6e2d62d3d9a"} Sep 08 10:36:31.308142 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.308095 2561 scope.go:117] "RemoveContainer" containerID="a6852a407519b1468b708ff334798a7698702fd5014c5f9029fe15e004091117" Sep 08 10:36:31.308332 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.308254 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-n5stc" Sep 08 10:36:31.310818 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.310790 2561 generic.go:356] "Generic (PLEG): container finished" podID="5e87bd1c-379f-48a5-afa2-e412963bb7b6" containerID="7f5f50e16fb84b40ab4002d3ed06c35649e2705efe16a4a3f5073a7c78a9540d" exitCode=0 Sep 08 10:36:31.310975 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.310908 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-xcx5s" event={"ID":"5e87bd1c-379f-48a5-afa2-e412963bb7b6","Type":"ContainerDied","Data":"7f5f50e16fb84b40ab4002d3ed06c35649e2705efe16a4a3f5073a7c78a9540d"} Sep 08 10:36:31.310975 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.310939 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-xcx5s" event={"ID":"5e87bd1c-379f-48a5-afa2-e412963bb7b6","Type":"ContainerDied","Data":"9f84b61fe48967771f9c8327e8b9089eb1a09761336ac8a2160a2c05730836b6"} Sep 08 10:36:31.310975 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.310959 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-xcx5s" Sep 08 10:36:31.321661 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.321623 2561 scope.go:117] "RemoveContainer" containerID="a6852a407519b1468b708ff334798a7698702fd5014c5f9029fe15e004091117" Sep 08 10:36:31.322190 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:36:31.322044 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6852a407519b1468b708ff334798a7698702fd5014c5f9029fe15e004091117\": container with ID starting with a6852a407519b1468b708ff334798a7698702fd5014c5f9029fe15e004091117 not found: ID does not exist" containerID="a6852a407519b1468b708ff334798a7698702fd5014c5f9029fe15e004091117" Sep 08 10:36:31.322190 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.322083 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6852a407519b1468b708ff334798a7698702fd5014c5f9029fe15e004091117"} err="failed to get container status \"a6852a407519b1468b708ff334798a7698702fd5014c5f9029fe15e004091117\": rpc error: code = NotFound desc = could not find container \"a6852a407519b1468b708ff334798a7698702fd5014c5f9029fe15e004091117\": container with ID starting with a6852a407519b1468b708ff334798a7698702fd5014c5f9029fe15e004091117 not found: ID does not exist" Sep 08 10:36:31.322190 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.322106 2561 scope.go:117] "RemoveContainer" containerID="7f5f50e16fb84b40ab4002d3ed06c35649e2705efe16a4a3f5073a7c78a9540d" Sep 08 10:36:31.336113 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.336073 2561 scope.go:117] "RemoveContainer" containerID="7f5f50e16fb84b40ab4002d3ed06c35649e2705efe16a4a3f5073a7c78a9540d" Sep 08 10:36:31.336960 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:36:31.336928 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f5f50e16fb84b40ab4002d3ed06c35649e2705efe16a4a3f5073a7c78a9540d\": container with ID starting with 7f5f50e16fb84b40ab4002d3ed06c35649e2705efe16a4a3f5073a7c78a9540d not found: ID does not exist" containerID="7f5f50e16fb84b40ab4002d3ed06c35649e2705efe16a4a3f5073a7c78a9540d" Sep 08 10:36:31.337089 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.336971 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f5f50e16fb84b40ab4002d3ed06c35649e2705efe16a4a3f5073a7c78a9540d"} err="failed to get container status \"7f5f50e16fb84b40ab4002d3ed06c35649e2705efe16a4a3f5073a7c78a9540d\": rpc error: code = NotFound desc = could not find container \"7f5f50e16fb84b40ab4002d3ed06c35649e2705efe16a4a3f5073a7c78a9540d\": container with ID starting with 7f5f50e16fb84b40ab4002d3ed06c35649e2705efe16a4a3f5073a7c78a9540d not found: ID does not exist" Sep 08 10:36:31.425790 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.425699 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6rdv\" (UniqueName: \"kubernetes.io/projected/5e87bd1c-379f-48a5-afa2-e412963bb7b6-kube-api-access-l6rdv\") pod \"5e87bd1c-379f-48a5-afa2-e412963bb7b6\" (UID: \"5e87bd1c-379f-48a5-afa2-e412963bb7b6\") " Sep 08 10:36:31.425790 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.425771 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h94h9\" (UniqueName: \"kubernetes.io/projected/fad98518-bf57-47ab-b0cc-c7cfb2550251-kube-api-access-h94h9\") pod \"fad98518-bf57-47ab-b0cc-c7cfb2550251\" (UID: \"fad98518-bf57-47ab-b0cc-c7cfb2550251\") " Sep 08 10:36:31.425992 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.425841 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/fad98518-bf57-47ab-b0cc-c7cfb2550251-postgresdata\") pod \"fad98518-bf57-47ab-b0cc-c7cfb2550251\" (UID: \"fad98518-bf57-47ab-b0cc-c7cfb2550251\") " Sep 08 10:36:31.428023 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.427985 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fad98518-bf57-47ab-b0cc-c7cfb2550251-kube-api-access-h94h9" (OuterVolumeSpecName: "kube-api-access-h94h9") pod "fad98518-bf57-47ab-b0cc-c7cfb2550251" (UID: "fad98518-bf57-47ab-b0cc-c7cfb2550251"). InnerVolumeSpecName "kube-api-access-h94h9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:36:31.428023 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.428016 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e87bd1c-379f-48a5-afa2-e412963bb7b6-kube-api-access-l6rdv" (OuterVolumeSpecName: "kube-api-access-l6rdv") pod "5e87bd1c-379f-48a5-afa2-e412963bb7b6" (UID: "5e87bd1c-379f-48a5-afa2-e412963bb7b6"). InnerVolumeSpecName "kube-api-access-l6rdv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:36:31.454674 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.454624 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad98518-bf57-47ab-b0cc-c7cfb2550251-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "fad98518-bf57-47ab-b0cc-c7cfb2550251" (UID: "fad98518-bf57-47ab-b0cc-c7cfb2550251"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:36:31.527175 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.527131 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6rdv\" (UniqueName: \"kubernetes.io/projected/5e87bd1c-379f-48a5-afa2-e412963bb7b6-kube-api-access-l6rdv\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:36:31.527175 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.527174 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h94h9\" (UniqueName: \"kubernetes.io/projected/fad98518-bf57-47ab-b0cc-c7cfb2550251-kube-api-access-h94h9\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:36:31.527391 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.527192 2561 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/fad98518-bf57-47ab-b0cc-c7cfb2550251-postgresdata\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:36:31.633787 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.633752 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-n5stc"] Sep 08 10:36:31.640365 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.640334 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-n5stc"] Sep 08 10:36:31.650897 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.650866 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-xcx5s"] Sep 08 10:36:31.655577 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:31.655551 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-xcx5s"] Sep 08 10:36:32.145273 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:32.145237 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e87bd1c-379f-48a5-afa2-e412963bb7b6" path="/var/lib/kubelet/pods/5e87bd1c-379f-48a5-afa2-e412963bb7b6/volumes" Sep 08 10:36:32.145620 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:32.145606 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fad98518-bf57-47ab-b0cc-c7cfb2550251" path="/var/lib/kubelet/pods/fad98518-bf57-47ab-b0cc-c7cfb2550251/volumes" Sep 08 10:36:37.075372 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:37.075319 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Sep 08 10:36:40.894942 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:40.894895 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="online" probeResult="failure" output="Get \"https://10.132.0.36:6567/health\": dial tcp 10.132.0.36:6567: connect: connection refused" Sep 08 10:36:47.075429 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:47.075379 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Sep 08 10:36:50.894915 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:50.894863 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="online" probeResult="failure" output="Get \"https://10.132.0.36:6567/health\": dial tcp 10.132.0.36:6567: connect: connection refused" Sep 08 10:36:50.895350 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:50.894986 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:36:57.075013 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:57.074954 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Sep 08 10:36:57.075408 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:57.075126 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:36:59.437208 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.437169 2561 generic.go:356] "Generic (PLEG): container finished" podID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerID="dc9d4b66e96950d5ac34ea767a62bd0e35d558eef0ec058a3323cb4483c1e3c9" exitCode=137 Sep 08 10:36:59.437671 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.437242 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" event={"ID":"3c9527d9-0a81-4e05-bb4d-3415952554f9","Type":"ContainerDied","Data":"dc9d4b66e96950d5ac34ea767a62bd0e35d558eef0ec058a3323cb4483c1e3c9"} Sep 08 10:36:59.439917 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.439888 2561 generic.go:356] "Generic (PLEG): container finished" podID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerID="34b0df01e123359bce1c9cef1c7b21bfc65d74d9f63a890f46702cfd48997367" exitCode=137 Sep 08 10:36:59.440064 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.439960 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" event={"ID":"83f0fd7a-68bf-46b0-89d3-38f18c26cdde","Type":"ContainerDied","Data":"34b0df01e123359bce1c9cef1c7b21bfc65d74d9f63a890f46702cfd48997367"} Sep 08 10:36:59.890032 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.890000 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:36:59.892301 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.892280 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:36:59.982347 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.982250 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tljn9\" (UniqueName: \"kubernetes.io/projected/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-kube-api-access-tljn9\") pod \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " Sep 08 10:36:59.982347 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.982306 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-feast-data\") pod \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " Sep 08 10:36:59.982347 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.982331 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-registry-tls\") pod \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " Sep 08 10:36:59.982825 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.982355 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-online-tls\") pod \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\" (UID: \"83f0fd7a-68bf-46b0-89d3-38f18c26cdde\") " Sep 08 10:36:59.982825 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.982385 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpj69\" (UniqueName: \"kubernetes.io/projected/3c9527d9-0a81-4e05-bb4d-3415952554f9-kube-api-access-cpj69\") pod \"3c9527d9-0a81-4e05-bb4d-3415952554f9\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " Sep 08 10:36:59.982825 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.982418 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3c9527d9-0a81-4e05-bb4d-3415952554f9-registry-tls\") pod \"3c9527d9-0a81-4e05-bb4d-3415952554f9\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " Sep 08 10:36:59.982825 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.982481 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3c9527d9-0a81-4e05-bb4d-3415952554f9-online-tls\") pod \"3c9527d9-0a81-4e05-bb4d-3415952554f9\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " Sep 08 10:36:59.982825 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.982512 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3c9527d9-0a81-4e05-bb4d-3415952554f9-feast-data\") pod \"3c9527d9-0a81-4e05-bb4d-3415952554f9\" (UID: \"3c9527d9-0a81-4e05-bb4d-3415952554f9\") " Sep 08 10:36:59.983278 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.983248 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9527d9-0a81-4e05-bb4d-3415952554f9-feast-data" (OuterVolumeSpecName: "feast-data") pod "3c9527d9-0a81-4e05-bb4d-3415952554f9" (UID: "3c9527d9-0a81-4e05-bb4d-3415952554f9"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:36:59.984727 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.984693 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-online-tls" (OuterVolumeSpecName: "online-tls") pod "83f0fd7a-68bf-46b0-89d3-38f18c26cdde" (UID: "83f0fd7a-68bf-46b0-89d3-38f18c26cdde"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:36:59.985100 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.985077 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "83f0fd7a-68bf-46b0-89d3-38f18c26cdde" (UID: "83f0fd7a-68bf-46b0-89d3-38f18c26cdde"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:36:59.985186 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.985113 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-kube-api-access-tljn9" (OuterVolumeSpecName: "kube-api-access-tljn9") pod "83f0fd7a-68bf-46b0-89d3-38f18c26cdde" (UID: "83f0fd7a-68bf-46b0-89d3-38f18c26cdde"). InnerVolumeSpecName "kube-api-access-tljn9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:36:59.985186 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.985151 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c9527d9-0a81-4e05-bb4d-3415952554f9-kube-api-access-cpj69" (OuterVolumeSpecName: "kube-api-access-cpj69") pod "3c9527d9-0a81-4e05-bb4d-3415952554f9" (UID: "3c9527d9-0a81-4e05-bb4d-3415952554f9"). InnerVolumeSpecName "kube-api-access-cpj69". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:36:59.985289 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.985269 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c9527d9-0a81-4e05-bb4d-3415952554f9-online-tls" (OuterVolumeSpecName: "online-tls") pod "3c9527d9-0a81-4e05-bb4d-3415952554f9" (UID: "3c9527d9-0a81-4e05-bb4d-3415952554f9"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:36:59.985337 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.985313 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c9527d9-0a81-4e05-bb4d-3415952554f9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3c9527d9-0a81-4e05-bb4d-3415952554f9" (UID: "3c9527d9-0a81-4e05-bb4d-3415952554f9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 10:36:59.993095 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:36:59.993063 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-feast-data" (OuterVolumeSpecName: "feast-data") pod "83f0fd7a-68bf-46b0-89d3-38f18c26cdde" (UID: "83f0fd7a-68bf-46b0-89d3-38f18c26cdde"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:37:00.083848 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.083793 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpj69\" (UniqueName: \"kubernetes.io/projected/3c9527d9-0a81-4e05-bb4d-3415952554f9-kube-api-access-cpj69\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:37:00.083848 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.083842 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3c9527d9-0a81-4e05-bb4d-3415952554f9-registry-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:37:00.083848 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.083856 2561 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3c9527d9-0a81-4e05-bb4d-3415952554f9-online-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:37:00.083848 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.083864 2561 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3c9527d9-0a81-4e05-bb4d-3415952554f9-feast-data\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:37:00.084102 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.083872 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tljn9\" (UniqueName: \"kubernetes.io/projected/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-kube-api-access-tljn9\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:37:00.084102 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.083882 2561 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-feast-data\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:37:00.084102 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.083890 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-registry-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:37:00.084102 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.083898 2561 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/83f0fd7a-68bf-46b0-89d3-38f18c26cdde-online-tls\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:37:00.447231 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.447198 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" Sep 08 10:37:00.447730 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.447201 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94" event={"ID":"3c9527d9-0a81-4e05-bb4d-3415952554f9","Type":"ContainerDied","Data":"9048b3bb229535f97abe8bec9e0fb3efdd8af5edd546afb0a18909dd84d88a92"} Sep 08 10:37:00.447730 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.447358 2561 scope.go:117] "RemoveContainer" containerID="ccd183419451b6ed4d6ddc356a5eacca076fc20ed1d0d8bfd44e6568a28e1533" Sep 08 10:37:00.449984 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.449958 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" event={"ID":"83f0fd7a-68bf-46b0-89d3-38f18c26cdde","Type":"ContainerDied","Data":"3cf0f65d75ba19d38c1b97ecb2f12e18e10fc8859a2725c6a5971920616251fa"} Sep 08 10:37:00.450090 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.450004 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246" Sep 08 10:37:00.457620 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.457592 2561 scope.go:117] "RemoveContainer" containerID="dc9d4b66e96950d5ac34ea767a62bd0e35d558eef0ec058a3323cb4483c1e3c9" Sep 08 10:37:00.466573 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.466543 2561 scope.go:117] "RemoveContainer" containerID="7399688fa40a30a7c03669ccee97a98b8a2ecb9796654976fa7f68d6b22e7247" Sep 08 10:37:00.472375 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.472344 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94"] Sep 08 10:37:00.476076 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.476051 2561 scope.go:117] "RemoveContainer" containerID="9713f076ffd4c7652515fffced4baa16342febff3f877ee6cdd42c3c0e8c4204" Sep 08 10:37:00.477007 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.476984 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7986c57686-64h94"] Sep 08 10:37:00.484702 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.484683 2561 scope.go:117] "RemoveContainer" containerID="f6d48d31bc5cb104d10b040e4914864b1e4cd7b53364af6aaedee025ac0acbbb" Sep 08 10:37:00.490088 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.490061 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246"] Sep 08 10:37:00.495985 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.495962 2561 scope.go:117] "RemoveContainer" containerID="34b0df01e123359bce1c9cef1c7b21bfc65d74d9f63a890f46702cfd48997367" Sep 08 10:37:00.496201 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.496182 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-fc449447c-n6246"] Sep 08 10:37:00.504955 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.504930 2561 scope.go:117] "RemoveContainer" containerID="7085ab85cfd433585fb1086cdb6178ee3228612897ed49bedba2ef9786800268" Sep 08 10:37:00.513186 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:00.513164 2561 scope.go:117] "RemoveContainer" containerID="9c2c56f208650852c707c7969d109e48e9f8396a6ddca9415d2b6ecf37df2da7" Sep 08 10:37:02.152787 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:02.152744 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" path="/var/lib/kubelet/pods/3c9527d9-0a81-4e05-bb4d-3415952554f9/volumes" Sep 08 10:37:02.153465 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:37:02.153448 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" path="/var/lib/kubelet/pods/83f0fd7a-68bf-46b0-89d3-38f18c26cdde/volumes" Sep 08 10:48:35.991987 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.991908 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mrlpt/must-gather-t7674"] Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992234 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81540954-a1ae-44d3-88b5-43f2d0c9d186" containerName="manager" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992245 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="81540954-a1ae-44d3-88b5-43f2d0c9d186" containerName="manager" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992254 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="feast-apply" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992259 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="feast-apply" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992266 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e87bd1c-379f-48a5-afa2-e412963bb7b6" containerName="redis" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992271 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e87bd1c-379f-48a5-afa2-e412963bb7b6" containerName="redis" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992286 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="online" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992291 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="online" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992300 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="feast-init" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992305 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="feast-init" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992311 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="registry" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992316 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="registry" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992324 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="online" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992329 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="online" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992336 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="feast-apply" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992341 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="feast-apply" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992347 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="feast-init" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992352 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="feast-init" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992359 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad98518-bf57-47ab-b0cc-c7cfb2550251" containerName="postgres" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992364 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad98518-bf57-47ab-b0cc-c7cfb2550251" containerName="postgres" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992372 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="registry" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992377 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="registry" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992425 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="online" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992432 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="registry" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992437 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="83f0fd7a-68bf-46b0-89d3-38f18c26cdde" containerName="registry" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992443 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="81540954-a1ae-44d3-88b5-43f2d0c9d186" containerName="manager" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992448 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c9527d9-0a81-4e05-bb4d-3415952554f9" containerName="online" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992454 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fad98518-bf57-47ab-b0cc-c7cfb2550251" containerName="postgres" Sep 08 10:48:35.992444 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.992461 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e87bd1c-379f-48a5-afa2-e412963bb7b6" containerName="redis" Sep 08 10:48:35.995502 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:35.995486 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mrlpt/must-gather-t7674" Sep 08 10:48:36.000845 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:36.000818 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mrlpt\"/\"openshift-service-ca.crt\"" Sep 08 10:48:36.001497 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:36.001469 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mrlpt\"/\"default-dockercfg-kjxjq\"" Sep 08 10:48:36.001497 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:36.001490 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mrlpt\"/\"kube-root-ca.crt\"" Sep 08 10:48:36.009427 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:36.009408 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mrlpt/must-gather-t7674"] Sep 08 10:48:36.055702 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:36.055679 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/40881ab4-ae28-4562-ac6a-3aab6f9abcd0-must-gather-output\") pod \"must-gather-t7674\" (UID: \"40881ab4-ae28-4562-ac6a-3aab6f9abcd0\") " pod="openshift-must-gather-mrlpt/must-gather-t7674" Sep 08 10:48:36.055812 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:36.055717 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92r9w\" (UniqueName: \"kubernetes.io/projected/40881ab4-ae28-4562-ac6a-3aab6f9abcd0-kube-api-access-92r9w\") pod \"must-gather-t7674\" (UID: \"40881ab4-ae28-4562-ac6a-3aab6f9abcd0\") " pod="openshift-must-gather-mrlpt/must-gather-t7674" Sep 08 10:48:36.156492 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:36.156471 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92r9w\" (UniqueName: \"kubernetes.io/projected/40881ab4-ae28-4562-ac6a-3aab6f9abcd0-kube-api-access-92r9w\") pod \"must-gather-t7674\" (UID: \"40881ab4-ae28-4562-ac6a-3aab6f9abcd0\") " pod="openshift-must-gather-mrlpt/must-gather-t7674" Sep 08 10:48:36.156616 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:36.156544 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/40881ab4-ae28-4562-ac6a-3aab6f9abcd0-must-gather-output\") pod \"must-gather-t7674\" (UID: \"40881ab4-ae28-4562-ac6a-3aab6f9abcd0\") " pod="openshift-must-gather-mrlpt/must-gather-t7674" Sep 08 10:48:36.156780 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:36.156763 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/40881ab4-ae28-4562-ac6a-3aab6f9abcd0-must-gather-output\") pod \"must-gather-t7674\" (UID: \"40881ab4-ae28-4562-ac6a-3aab6f9abcd0\") " pod="openshift-must-gather-mrlpt/must-gather-t7674" Sep 08 10:48:36.166218 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:36.166195 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92r9w\" (UniqueName: \"kubernetes.io/projected/40881ab4-ae28-4562-ac6a-3aab6f9abcd0-kube-api-access-92r9w\") pod \"must-gather-t7674\" (UID: \"40881ab4-ae28-4562-ac6a-3aab6f9abcd0\") " pod="openshift-must-gather-mrlpt/must-gather-t7674" Sep 08 10:48:36.318004 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:36.317952 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mrlpt/must-gather-t7674" Sep 08 10:48:36.587083 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:36.586856 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mrlpt/must-gather-t7674"] Sep 08 10:48:36.589891 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:48:36.589863 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40881ab4_ae28_4562_ac6a_3aab6f9abcd0.slice/crio-e07619093b96ff3edfcd9052108076686352db49a6c46dd4e7f713db6558c203 WatchSource:0}: Error finding container e07619093b96ff3edfcd9052108076686352db49a6c46dd4e7f713db6558c203: Status 404 returned error can't find the container with id e07619093b96ff3edfcd9052108076686352db49a6c46dd4e7f713db6558c203 Sep 08 10:48:36.591713 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:36.591694 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 10:48:37.221013 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:37.220978 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mrlpt/must-gather-t7674" event={"ID":"40881ab4-ae28-4562-ac6a-3aab6f9abcd0","Type":"ContainerStarted","Data":"e07619093b96ff3edfcd9052108076686352db49a6c46dd4e7f713db6558c203"} Sep 08 10:48:43.251562 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:43.251512 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mrlpt/must-gather-t7674" event={"ID":"40881ab4-ae28-4562-ac6a-3aab6f9abcd0","Type":"ContainerStarted","Data":"e6ad6bef1ba93b96e460f619e1bf5b5e97bc34a937833b9c758e089de3cd0827"} Sep 08 10:48:43.251562 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:43.251572 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mrlpt/must-gather-t7674" event={"ID":"40881ab4-ae28-4562-ac6a-3aab6f9abcd0","Type":"ContainerStarted","Data":"b70d103293952b82d9d30f2358e284a95972d575c0b8601c42d668b44fd6a0d4"} Sep 08 10:48:43.273751 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:43.273693 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mrlpt/must-gather-t7674" podStartSLOduration=2.146545219 podStartE2EDuration="8.273677298s" totalImagesPullingTime="6.127132079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:48:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:48:43.272295789 +0000 UTC m=+1899.641561087" watchObservedRunningTime="2026-09-08 10:48:43.273677298 +0000 UTC m=+1899.642942594" Sep 08 10:48:53.294888 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:53.294848 2561 generic.go:356] "Generic (PLEG): container finished" podID="40881ab4-ae28-4562-ac6a-3aab6f9abcd0" containerID="b70d103293952b82d9d30f2358e284a95972d575c0b8601c42d668b44fd6a0d4" exitCode=0 Sep 08 10:48:53.295424 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:53.294904 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mrlpt/must-gather-t7674" event={"ID":"40881ab4-ae28-4562-ac6a-3aab6f9abcd0","Type":"ContainerDied","Data":"b70d103293952b82d9d30f2358e284a95972d575c0b8601c42d668b44fd6a0d4"} Sep 08 10:48:53.295424 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:53.295286 2561 scope.go:117] "RemoveContainer" containerID="b70d103293952b82d9d30f2358e284a95972d575c0b8601c42d668b44fd6a0d4" Sep 08 10:48:53.361971 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:53.361944 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-mrlpt_must-gather-t7674_40881ab4-ae28-4562-ac6a-3aab6f9abcd0/gather/0.log" Sep 08 10:48:53.825310 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:53.825274 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rvr9p/must-gather-v4wmj"] Sep 08 10:48:53.828425 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:53.828404 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rvr9p/must-gather-v4wmj" Sep 08 10:48:53.831512 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:53.831485 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rvr9p\"/\"default-dockercfg-dmg7j\"" Sep 08 10:48:53.832350 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:53.832323 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rvr9p\"/\"openshift-service-ca.crt\"" Sep 08 10:48:53.832350 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:53.832334 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rvr9p\"/\"kube-root-ca.crt\"" Sep 08 10:48:53.837439 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:53.837417 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rvr9p/must-gather-v4wmj"] Sep 08 10:48:53.911244 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:53.911210 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pwvt\" (UniqueName: \"kubernetes.io/projected/945bddfd-241b-48e2-9da6-22d58571f607-kube-api-access-8pwvt\") pod \"must-gather-v4wmj\" (UID: \"945bddfd-241b-48e2-9da6-22d58571f607\") " pod="openshift-must-gather-rvr9p/must-gather-v4wmj" Sep 08 10:48:53.911402 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:53.911253 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/945bddfd-241b-48e2-9da6-22d58571f607-must-gather-output\") pod \"must-gather-v4wmj\" (UID: \"945bddfd-241b-48e2-9da6-22d58571f607\") " pod="openshift-must-gather-rvr9p/must-gather-v4wmj" Sep 08 10:48:54.011983 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:54.011942 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pwvt\" (UniqueName: \"kubernetes.io/projected/945bddfd-241b-48e2-9da6-22d58571f607-kube-api-access-8pwvt\") pod \"must-gather-v4wmj\" (UID: \"945bddfd-241b-48e2-9da6-22d58571f607\") " pod="openshift-must-gather-rvr9p/must-gather-v4wmj" Sep 08 10:48:54.012118 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:54.011990 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/945bddfd-241b-48e2-9da6-22d58571f607-must-gather-output\") pod \"must-gather-v4wmj\" (UID: \"945bddfd-241b-48e2-9da6-22d58571f607\") " pod="openshift-must-gather-rvr9p/must-gather-v4wmj" Sep 08 10:48:54.012283 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:54.012268 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/945bddfd-241b-48e2-9da6-22d58571f607-must-gather-output\") pod \"must-gather-v4wmj\" (UID: \"945bddfd-241b-48e2-9da6-22d58571f607\") " pod="openshift-must-gather-rvr9p/must-gather-v4wmj" Sep 08 10:48:54.026604 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:54.026578 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pwvt\" (UniqueName: \"kubernetes.io/projected/945bddfd-241b-48e2-9da6-22d58571f607-kube-api-access-8pwvt\") pod \"must-gather-v4wmj\" (UID: \"945bddfd-241b-48e2-9da6-22d58571f607\") " pod="openshift-must-gather-rvr9p/must-gather-v4wmj" Sep 08 10:48:54.137946 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:54.137867 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rvr9p/must-gather-v4wmj" Sep 08 10:48:54.416270 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:54.416187 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rvr9p/must-gather-v4wmj"] Sep 08 10:48:54.419022 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:48:54.418991 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod945bddfd_241b_48e2_9da6_22d58571f607.slice/crio-1caa583bc13eea824f6166bb973e9d342852df6b1192257c7d5773dcf3938aa6 WatchSource:0}: Error finding container 1caa583bc13eea824f6166bb973e9d342852df6b1192257c7d5773dcf3938aa6: Status 404 returned error can't find the container with id 1caa583bc13eea824f6166bb973e9d342852df6b1192257c7d5773dcf3938aa6 Sep 08 10:48:55.303668 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:55.303620 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rvr9p/must-gather-v4wmj" event={"ID":"945bddfd-241b-48e2-9da6-22d58571f607","Type":"ContainerStarted","Data":"1caa583bc13eea824f6166bb973e9d342852df6b1192257c7d5773dcf3938aa6"} Sep 08 10:48:56.310281 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:56.310232 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rvr9p/must-gather-v4wmj" event={"ID":"945bddfd-241b-48e2-9da6-22d58571f607","Type":"ContainerStarted","Data":"a02b93e3e8df641aaf54b93298bde7ec698ac6ab12216d85a9633260f1207c3e"} Sep 08 10:48:56.310798 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:56.310297 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rvr9p/must-gather-v4wmj" event={"ID":"945bddfd-241b-48e2-9da6-22d58571f607","Type":"ContainerStarted","Data":"5cd15210b7a7eadc4a9d85b53452006ae3d956be004d47fb45dbed89a9e0bcf0"} Sep 08 10:48:56.327875 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:56.327812 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rvr9p/must-gather-v4wmj" podStartSLOduration=1.943456383 podStartE2EDuration="3.327796051s" totalImagesPullingTime="1.384339668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 10:48:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:48:56.327203698 +0000 UTC m=+1912.696468995" watchObservedRunningTime="2026-09-08 10:48:56.327796051 +0000 UTC m=+1912.697061347" Sep 08 10:48:57.385513 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:57.385479 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-56snn_e0ef2303-d3cc-4560-b248-8d0c1b6e94e2/global-pull-secret-syncer/0.log" Sep 08 10:48:57.583289 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:57.583255 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qhxw7_63e77b90-9f35-4b51-a977-290598c60b83/konnectivity-agent/0.log" Sep 08 10:48:57.686280 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:57.686190 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-203.ec2.internal_f4f934834e3ed193d36ec5ddfb5f38fb/haproxy/0.log" Sep 08 10:48:58.673731 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:58.673684 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-mrlpt/must-gather-t7674"] Sep 08 10:48:58.674193 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:58.673975 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-mrlpt/must-gather-t7674" podUID="40881ab4-ae28-4562-ac6a-3aab6f9abcd0" containerName="copy" containerID="cri-o://e6ad6bef1ba93b96e460f619e1bf5b5e97bc34a937833b9c758e089de3cd0827" gracePeriod=2 Sep 08 10:48:58.683706 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:58.683663 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-mrlpt/must-gather-t7674"] Sep 08 10:48:59.075442 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.075366 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-mrlpt_must-gather-t7674_40881ab4-ae28-4562-ac6a-3aab6f9abcd0/copy/0.log" Sep 08 10:48:59.084931 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.084602 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mrlpt/must-gather-t7674" Sep 08 10:48:59.091444 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:48:59.089566 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-t7674\" is forbidden: User \"system:node:ip-10-0-132-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-mrlpt\": no relationship found between node 'ip-10-0-132-203.ec2.internal' and this object" podUID="40881ab4-ae28-4562-ac6a-3aab6f9abcd0" pod="openshift-must-gather-mrlpt/must-gather-t7674" Sep 08 10:48:59.166391 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.165927 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92r9w\" (UniqueName: \"kubernetes.io/projected/40881ab4-ae28-4562-ac6a-3aab6f9abcd0-kube-api-access-92r9w\") pod \"40881ab4-ae28-4562-ac6a-3aab6f9abcd0\" (UID: \"40881ab4-ae28-4562-ac6a-3aab6f9abcd0\") " Sep 08 10:48:59.166391 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.166108 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/40881ab4-ae28-4562-ac6a-3aab6f9abcd0-must-gather-output\") pod \"40881ab4-ae28-4562-ac6a-3aab6f9abcd0\" (UID: \"40881ab4-ae28-4562-ac6a-3aab6f9abcd0\") " Sep 08 10:48:59.166840 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.166812 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40881ab4-ae28-4562-ac6a-3aab6f9abcd0-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "40881ab4-ae28-4562-ac6a-3aab6f9abcd0" (UID: "40881ab4-ae28-4562-ac6a-3aab6f9abcd0"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 10:48:59.169387 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.169357 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40881ab4-ae28-4562-ac6a-3aab6f9abcd0-kube-api-access-92r9w" (OuterVolumeSpecName: "kube-api-access-92r9w") pod "40881ab4-ae28-4562-ac6a-3aab6f9abcd0" (UID: "40881ab4-ae28-4562-ac6a-3aab6f9abcd0"). InnerVolumeSpecName "kube-api-access-92r9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 10:48:59.267720 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.267674 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92r9w\" (UniqueName: \"kubernetes.io/projected/40881ab4-ae28-4562-ac6a-3aab6f9abcd0-kube-api-access-92r9w\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:48:59.267720 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.267718 2561 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/40881ab4-ae28-4562-ac6a-3aab6f9abcd0-must-gather-output\") on node \"ip-10-0-132-203.ec2.internal\" DevicePath \"\"" Sep 08 10:48:59.329600 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.329555 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-mrlpt_must-gather-t7674_40881ab4-ae28-4562-ac6a-3aab6f9abcd0/copy/0.log" Sep 08 10:48:59.330225 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.330192 2561 generic.go:356] "Generic (PLEG): container finished" podID="40881ab4-ae28-4562-ac6a-3aab6f9abcd0" containerID="e6ad6bef1ba93b96e460f619e1bf5b5e97bc34a937833b9c758e089de3cd0827" exitCode=143 Sep 08 10:48:59.330358 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.330352 2561 scope.go:117] "RemoveContainer" containerID="e6ad6bef1ba93b96e460f619e1bf5b5e97bc34a937833b9c758e089de3cd0827" Sep 08 10:48:59.330602 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.330583 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mrlpt/must-gather-t7674" Sep 08 10:48:59.339030 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:48:59.338993 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-t7674\" is forbidden: User \"system:node:ip-10-0-132-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-mrlpt\": no relationship found between node 'ip-10-0-132-203.ec2.internal' and this object" podUID="40881ab4-ae28-4562-ac6a-3aab6f9abcd0" pod="openshift-must-gather-mrlpt/must-gather-t7674" Sep 08 10:48:59.352750 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:48:59.352694 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-t7674\" is forbidden: User \"system:node:ip-10-0-132-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-mrlpt\": no relationship found between node 'ip-10-0-132-203.ec2.internal' and this object" podUID="40881ab4-ae28-4562-ac6a-3aab6f9abcd0" pod="openshift-must-gather-mrlpt/must-gather-t7674" Sep 08 10:48:59.358498 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.358066 2561 scope.go:117] "RemoveContainer" containerID="b70d103293952b82d9d30f2358e284a95972d575c0b8601c42d668b44fd6a0d4" Sep 08 10:48:59.384096 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.383751 2561 scope.go:117] "RemoveContainer" containerID="e6ad6bef1ba93b96e460f619e1bf5b5e97bc34a937833b9c758e089de3cd0827" Sep 08 10:48:59.384809 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:48:59.384347 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6ad6bef1ba93b96e460f619e1bf5b5e97bc34a937833b9c758e089de3cd0827\": container with ID starting with e6ad6bef1ba93b96e460f619e1bf5b5e97bc34a937833b9c758e089de3cd0827 not found: ID does not exist" containerID="e6ad6bef1ba93b96e460f619e1bf5b5e97bc34a937833b9c758e089de3cd0827" Sep 08 10:48:59.384809 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.384384 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6ad6bef1ba93b96e460f619e1bf5b5e97bc34a937833b9c758e089de3cd0827"} err="failed to get container status \"e6ad6bef1ba93b96e460f619e1bf5b5e97bc34a937833b9c758e089de3cd0827\": rpc error: code = NotFound desc = could not find container \"e6ad6bef1ba93b96e460f619e1bf5b5e97bc34a937833b9c758e089de3cd0827\": container with ID starting with e6ad6bef1ba93b96e460f619e1bf5b5e97bc34a937833b9c758e089de3cd0827 not found: ID does not exist" Sep 08 10:48:59.384809 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.384409 2561 scope.go:117] "RemoveContainer" containerID="b70d103293952b82d9d30f2358e284a95972d575c0b8601c42d668b44fd6a0d4" Sep 08 10:48:59.385156 ip-10-0-132-203 kubenswrapper[2561]: E0908 10:48:59.385090 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b70d103293952b82d9d30f2358e284a95972d575c0b8601c42d668b44fd6a0d4\": container with ID starting with b70d103293952b82d9d30f2358e284a95972d575c0b8601c42d668b44fd6a0d4 not found: ID does not exist" containerID="b70d103293952b82d9d30f2358e284a95972d575c0b8601c42d668b44fd6a0d4" Sep 08 10:48:59.385156 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:48:59.385117 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b70d103293952b82d9d30f2358e284a95972d575c0b8601c42d668b44fd6a0d4"} err="failed to get container status \"b70d103293952b82d9d30f2358e284a95972d575c0b8601c42d668b44fd6a0d4\": rpc error: code = NotFound desc = could not find container \"b70d103293952b82d9d30f2358e284a95972d575c0b8601c42d668b44fd6a0d4\": container with ID starting with b70d103293952b82d9d30f2358e284a95972d575c0b8601c42d668b44fd6a0d4 not found: ID does not exist" Sep 08 10:49:00.150359 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:00.150322 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40881ab4-ae28-4562-ac6a-3aab6f9abcd0" path="/var/lib/kubelet/pods/40881ab4-ae28-4562-ac6a-3aab6f9abcd0/volumes" Sep 08 10:49:01.902234 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:01.902153 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1437588c-3afd-463c-9217-8cef10e86cca/alertmanager/0.log" Sep 08 10:49:01.934764 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:01.934728 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1437588c-3afd-463c-9217-8cef10e86cca/config-reloader/0.log" Sep 08 10:49:01.964584 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:01.964546 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1437588c-3afd-463c-9217-8cef10e86cca/kube-rbac-proxy-web/0.log" Sep 08 10:49:01.992957 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:01.992924 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1437588c-3afd-463c-9217-8cef10e86cca/kube-rbac-proxy/0.log" Sep 08 10:49:02.035475 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:02.035425 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1437588c-3afd-463c-9217-8cef10e86cca/kube-rbac-proxy-metric/0.log" Sep 08 10:49:02.062070 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:02.062040 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1437588c-3afd-463c-9217-8cef10e86cca/prom-label-proxy/0.log" Sep 08 10:49:02.088868 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:02.088839 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1437588c-3afd-463c-9217-8cef10e86cca/init-config-reloader/0.log" Sep 08 10:49:02.170811 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:02.170725 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hjzj2_3a437240-6ceb-4a5f-b994-4119492c928f/kube-state-metrics/0.log" Sep 08 10:49:02.196839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:02.196811 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hjzj2_3a437240-6ceb-4a5f-b994-4119492c928f/kube-rbac-proxy-main/0.log" Sep 08 10:49:02.225105 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:02.225057 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-hjzj2_3a437240-6ceb-4a5f-b994-4119492c928f/kube-rbac-proxy-self/0.log" Sep 08 10:49:02.258895 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:02.258844 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6fcd85b9b8-q85w4_01e8dec6-55f6-4a91-bff3-a8ccf61e19bc/metrics-server/0.log" Sep 08 10:49:02.431753 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:02.431646 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8299s_cf550904-a7ea-4cd1-8843-766b17279143/node-exporter/0.log" Sep 08 10:49:02.458613 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:02.458576 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8299s_cf550904-a7ea-4cd1-8843-766b17279143/kube-rbac-proxy/0.log" Sep 08 10:49:02.485847 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:02.485814 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8299s_cf550904-a7ea-4cd1-8843-766b17279143/init-textfile/0.log" Sep 08 10:49:02.615797 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:02.615767 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-t7bpc_97778083-73ca-43eb-9454-1d3a28e8528a/kube-rbac-proxy-main/0.log" Sep 08 10:49:02.645308 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:02.645280 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-t7bpc_97778083-73ca-43eb-9454-1d3a28e8528a/kube-rbac-proxy-self/0.log" Sep 08 10:49:02.681254 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:02.681226 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-t7bpc_97778083-73ca-43eb-9454-1d3a28e8528a/openshift-state-metrics/0.log" Sep 08 10:49:03.001220 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:03.001176 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-c9cp8_6fd9e1e2-cfae-4be8-a82c-bea62267a9ad/prometheus-operator/0.log" Sep 08 10:49:03.024746 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:03.024659 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-c9cp8_6fd9e1e2-cfae-4be8-a82c-bea62267a9ad/kube-rbac-proxy/0.log" Sep 08 10:49:03.057128 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:03.057095 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-kzzx6_c9ce2aaf-a9c4-433c-9747-40228ed457ab/prometheus-operator-admission-webhook/0.log" Sep 08 10:49:03.096703 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:03.096667 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54bf4db66-fst9v_f0e73e10-7e61-40b7-935e-bedd4b1e481e/telemeter-client/0.log" Sep 08 10:49:03.125929 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:03.125891 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54bf4db66-fst9v_f0e73e10-7e61-40b7-935e-bedd4b1e481e/reload/0.log" Sep 08 10:49:03.155652 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:03.155613 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54bf4db66-fst9v_f0e73e10-7e61-40b7-935e-bedd4b1e481e/kube-rbac-proxy/0.log" Sep 08 10:49:03.191238 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:03.191207 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7697b4fdb7-76k8c_91f35eca-0c71-4f67-b041-3fffa2d7c256/thanos-query/0.log" Sep 08 10:49:03.221431 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:03.221398 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7697b4fdb7-76k8c_91f35eca-0c71-4f67-b041-3fffa2d7c256/kube-rbac-proxy-web/0.log" Sep 08 10:49:03.252284 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:03.252244 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7697b4fdb7-76k8c_91f35eca-0c71-4f67-b041-3fffa2d7c256/kube-rbac-proxy/0.log" Sep 08 10:49:03.283614 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:03.283501 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7697b4fdb7-76k8c_91f35eca-0c71-4f67-b041-3fffa2d7c256/prom-label-proxy/0.log" Sep 08 10:49:03.312566 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:03.312515 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7697b4fdb7-76k8c_91f35eca-0c71-4f67-b041-3fffa2d7c256/kube-rbac-proxy-rules/0.log" Sep 08 10:49:03.344689 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:03.344648 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7697b4fdb7-76k8c_91f35eca-0c71-4f67-b041-3fffa2d7c256/kube-rbac-proxy-metrics/0.log" Sep 08 10:49:04.847511 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:04.847467 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp"] Sep 08 10:49:04.848071 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:04.847984 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40881ab4-ae28-4562-ac6a-3aab6f9abcd0" containerName="gather" Sep 08 10:49:04.848071 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:04.848005 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="40881ab4-ae28-4562-ac6a-3aab6f9abcd0" containerName="gather" Sep 08 10:49:04.848071 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:04.848043 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40881ab4-ae28-4562-ac6a-3aab6f9abcd0" containerName="copy" Sep 08 10:49:04.848071 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:04.848052 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="40881ab4-ae28-4562-ac6a-3aab6f9abcd0" containerName="copy" Sep 08 10:49:04.848310 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:04.848151 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="40881ab4-ae28-4562-ac6a-3aab6f9abcd0" containerName="gather" Sep 08 10:49:04.848310 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:04.848169 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="40881ab4-ae28-4562-ac6a-3aab6f9abcd0" containerName="copy" Sep 08 10:49:04.853989 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:04.853955 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:04.868439 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:04.868395 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp"] Sep 08 10:49:05.038040 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.037989 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f302452f-09a4-41a5-9ded-30132c3f8afe-proc\") pod \"perf-node-gather-daemonset-kcbcp\" (UID: \"f302452f-09a4-41a5-9ded-30132c3f8afe\") " pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:05.038250 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.038052 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f302452f-09a4-41a5-9ded-30132c3f8afe-podres\") pod \"perf-node-gather-daemonset-kcbcp\" (UID: \"f302452f-09a4-41a5-9ded-30132c3f8afe\") " pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:05.038250 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.038184 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvjbj\" (UniqueName: \"kubernetes.io/projected/f302452f-09a4-41a5-9ded-30132c3f8afe-kube-api-access-bvjbj\") pod \"perf-node-gather-daemonset-kcbcp\" (UID: \"f302452f-09a4-41a5-9ded-30132c3f8afe\") " pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:05.038250 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.038239 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f302452f-09a4-41a5-9ded-30132c3f8afe-lib-modules\") pod \"perf-node-gather-daemonset-kcbcp\" (UID: \"f302452f-09a4-41a5-9ded-30132c3f8afe\") " pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:05.038420 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.038309 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f302452f-09a4-41a5-9ded-30132c3f8afe-sys\") pod \"perf-node-gather-daemonset-kcbcp\" (UID: \"f302452f-09a4-41a5-9ded-30132c3f8afe\") " pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:05.139095 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.138995 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvjbj\" (UniqueName: \"kubernetes.io/projected/f302452f-09a4-41a5-9ded-30132c3f8afe-kube-api-access-bvjbj\") pod \"perf-node-gather-daemonset-kcbcp\" (UID: \"f302452f-09a4-41a5-9ded-30132c3f8afe\") " pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:05.139095 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.139061 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f302452f-09a4-41a5-9ded-30132c3f8afe-lib-modules\") pod \"perf-node-gather-daemonset-kcbcp\" (UID: \"f302452f-09a4-41a5-9ded-30132c3f8afe\") " pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:05.139359 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.139120 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f302452f-09a4-41a5-9ded-30132c3f8afe-sys\") pod \"perf-node-gather-daemonset-kcbcp\" (UID: \"f302452f-09a4-41a5-9ded-30132c3f8afe\") " pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:05.139359 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.139160 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f302452f-09a4-41a5-9ded-30132c3f8afe-proc\") pod \"perf-node-gather-daemonset-kcbcp\" (UID: \"f302452f-09a4-41a5-9ded-30132c3f8afe\") " pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:05.139359 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.139190 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f302452f-09a4-41a5-9ded-30132c3f8afe-podres\") pod \"perf-node-gather-daemonset-kcbcp\" (UID: \"f302452f-09a4-41a5-9ded-30132c3f8afe\") " pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:05.140326 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.140295 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f302452f-09a4-41a5-9ded-30132c3f8afe-lib-modules\") pod \"perf-node-gather-daemonset-kcbcp\" (UID: \"f302452f-09a4-41a5-9ded-30132c3f8afe\") " pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:05.140463 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.140331 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f302452f-09a4-41a5-9ded-30132c3f8afe-proc\") pod \"perf-node-gather-daemonset-kcbcp\" (UID: \"f302452f-09a4-41a5-9ded-30132c3f8afe\") " pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:05.140463 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.140364 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f302452f-09a4-41a5-9ded-30132c3f8afe-podres\") pod \"perf-node-gather-daemonset-kcbcp\" (UID: \"f302452f-09a4-41a5-9ded-30132c3f8afe\") " pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:05.140463 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.140334 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f302452f-09a4-41a5-9ded-30132c3f8afe-sys\") pod \"perf-node-gather-daemonset-kcbcp\" (UID: \"f302452f-09a4-41a5-9ded-30132c3f8afe\") " pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:05.154286 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.154252 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvjbj\" (UniqueName: \"kubernetes.io/projected/f302452f-09a4-41a5-9ded-30132c3f8afe-kube-api-access-bvjbj\") pod \"perf-node-gather-daemonset-kcbcp\" (UID: \"f302452f-09a4-41a5-9ded-30132c3f8afe\") " pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:05.168260 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.168220 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:05.397838 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.397748 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d858685c7-vr45p_755502c6-7937-4732-91a3-1ccf11d99aae/console/0.log" Sep 08 10:49:05.437499 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.437465 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-7gv99_d605b050-e240-493c-8692-ca923917cdd6/download-server/0.log" Sep 08 10:49:05.458730 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:05.458678 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp"] Sep 08 10:49:05.461430 ip-10-0-132-203 kubenswrapper[2561]: W0908 10:49:05.461373 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf302452f_09a4_41a5_9ded_30132c3f8afe.slice/crio-537bc485f0a52bd5ea7c69d749e2a9e7a27445d647966b83ee30280ebce52f4c WatchSource:0}: Error finding container 537bc485f0a52bd5ea7c69d749e2a9e7a27445d647966b83ee30280ebce52f4c: Status 404 returned error can't find the container with id 537bc485f0a52bd5ea7c69d749e2a9e7a27445d647966b83ee30280ebce52f4c Sep 08 10:49:06.372450 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:06.372411 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" event={"ID":"f302452f-09a4-41a5-9ded-30132c3f8afe","Type":"ContainerStarted","Data":"924545bb4acf3252b0cc9c94a6fb7348fd6804d712371bc1b6231b5156f96c6a"} Sep 08 10:49:06.372880 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:06.372468 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" event={"ID":"f302452f-09a4-41a5-9ded-30132c3f8afe","Type":"ContainerStarted","Data":"537bc485f0a52bd5ea7c69d749e2a9e7a27445d647966b83ee30280ebce52f4c"} Sep 08 10:49:06.372880 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:06.372655 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:06.391577 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:06.391471 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" podStartSLOduration=2.391448092 podStartE2EDuration="2.391448092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 10:49:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 10:49:06.390886459 +0000 UTC m=+1922.760151757" watchObservedRunningTime="2026-09-08 10:49:06.391448092 +0000 UTC m=+1922.760713390" Sep 08 10:49:06.777849 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:06.777817 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8nhg6_d3689cda-0bc7-4455-8530-c685d3a0fe94/dns/0.log" Sep 08 10:49:06.816783 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:06.816751 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8nhg6_d3689cda-0bc7-4455-8530-c685d3a0fe94/kube-rbac-proxy/0.log" Sep 08 10:49:07.006893 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:07.006859 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lw9vz_e9f96208-ea87-4822-ae72-9c07e5d5fa8f/dns-node-resolver/0.log" Sep 08 10:49:07.512511 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:07.512480 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pv25g_919f3306-bc79-43be-92f0-2b46f301d6f7/node-ca/0.log" Sep 08 10:49:08.413472 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:08.413445 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64bb84f4d-q4pg8_7484ddef-5576-43fe-857a-000cb91c4460/router/0.log" Sep 08 10:49:08.784430 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:08.784400 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8dfxz_3c246116-fdc5-4fde-8f72-8835d9718f7d/serve-healthcheck-canary/0.log" Sep 08 10:49:09.349199 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:09.349051 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jxz2z_4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80/kube-rbac-proxy/0.log" Sep 08 10:49:09.381324 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:09.381294 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jxz2z_4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80/exporter/0.log" Sep 08 10:49:09.416486 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:09.416458 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jxz2z_4e9e96ff-b26f-40f3-ab4c-5a1bd4115a80/extractor/0.log" Sep 08 10:49:12.388572 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:12.388517 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rvr9p/perf-node-gather-daemonset-kcbcp" Sep 08 10:49:16.383312 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:16.383270 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4mxks_a910ad03-91d3-448e-bf66-56effa2459f3/kube-multus/0.log" Sep 08 10:49:16.437594 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:16.437557 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-552hj_6ee8913a-efe5-42f7-bf88-3a0439a828ba/kube-multus-additional-cni-plugins/0.log" Sep 08 10:49:16.464422 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:16.464298 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-552hj_6ee8913a-efe5-42f7-bf88-3a0439a828ba/egress-router-binary-copy/0.log" Sep 08 10:49:16.490964 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:16.490936 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-552hj_6ee8913a-efe5-42f7-bf88-3a0439a828ba/cni-plugins/0.log" Sep 08 10:49:16.516870 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:16.516836 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-552hj_6ee8913a-efe5-42f7-bf88-3a0439a828ba/bond-cni-plugin/0.log" Sep 08 10:49:16.542387 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:16.542361 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-552hj_6ee8913a-efe5-42f7-bf88-3a0439a828ba/routeoverride-cni/0.log" Sep 08 10:49:16.567806 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:16.567776 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-552hj_6ee8913a-efe5-42f7-bf88-3a0439a828ba/whereabouts-cni-bincopy/0.log" Sep 08 10:49:16.591671 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:16.591643 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-552hj_6ee8913a-efe5-42f7-bf88-3a0439a828ba/whereabouts-cni/0.log" Sep 08 10:49:17.091663 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:17.091633 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pr8g9_db4c4c5b-19b6-439a-bcd2-0631f7626479/network-metrics-daemon/0.log" Sep 08 10:49:17.123500 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:17.123468 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pr8g9_db4c4c5b-19b6-439a-bcd2-0631f7626479/kube-rbac-proxy/0.log" Sep 08 10:49:18.516839 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:18.516811 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wx64r_d7f1ba8a-c265-4a99-ae50-00c5b2b3c217/ovn-controller/0.log" Sep 08 10:49:18.537786 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:18.537751 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wx64r_d7f1ba8a-c265-4a99-ae50-00c5b2b3c217/ovn-acl-logging/0.log" Sep 08 10:49:18.564987 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:18.564961 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wx64r_d7f1ba8a-c265-4a99-ae50-00c5b2b3c217/kube-rbac-proxy-node/0.log" Sep 08 10:49:18.592784 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:18.592754 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wx64r_d7f1ba8a-c265-4a99-ae50-00c5b2b3c217/kube-rbac-proxy-ovn-metrics/0.log" Sep 08 10:49:18.615254 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:18.615217 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wx64r_d7f1ba8a-c265-4a99-ae50-00c5b2b3c217/northd/0.log" Sep 08 10:49:18.639621 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:18.639594 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wx64r_d7f1ba8a-c265-4a99-ae50-00c5b2b3c217/nbdb/0.log" Sep 08 10:49:18.663077 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:18.663050 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wx64r_d7f1ba8a-c265-4a99-ae50-00c5b2b3c217/sbdb/0.log" Sep 08 10:49:18.710860 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:18.710823 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wx64r_d7f1ba8a-c265-4a99-ae50-00c5b2b3c217/ovnkube-controller/0.log" Sep 08 10:49:19.794868 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:19.794837 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-xlksv_0d5943f3-ba9e-4dc9-91c1-9787f83d5305/network-check-target-container/0.log" Sep 08 10:49:20.844233 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:20.844201 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-wkwnr_4425cf1f-0af5-4990-9169-4ce1848e739d/iptables-alerter/0.log" Sep 08 10:49:21.579335 ip-10-0-132-203 kubenswrapper[2561]: I0908 10:49:21.579303 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-lsnsb_08fd25ee-4cf2-467f-aa66-cb3f454c528c/tuned/0.log"